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Foreword 



The Nation relies on federally funded research to address many national objectives. With 
global competition, changing student d«. ^nographics, rising demands for research funds, and 
the prospect of constricted budgets, Congress and the executive branch must make difficult 
choices in supporting U.S. science and engineering. The House Committee on Science, Space, 
and Technology asked OTA to examine the Federal research system— a conglomeration of 
many separate systems that sponscn:, oversee, and perfomi research — and the challenges that 
it will face in the 1990s. 

Given the exceptional history, strength, and character of U.S. research, there will always 
be more opportunities than can be funded, more (Reserving researchers competing than can be 
sustained, and more institutions seeking to expand than the prime sponsor — 'he Federal 
Government — can fund. The objective for government, ±en, is to ensure continued funding 
for a full portfolio of first-rate research and a high^aliber research work force to assure 
long-term scientific progress. This report analyzes what OTA identifies as four pressing 
challenges for the research system in the 1990s: setting priorities in funding, understanding 
trends in research expenditures, preparing human resources for the future research work force, 
and supplying appropriate data for ongoing research decisionmaking. Managing the Federal 
research system requires more than funding; it means devising ways to retain the diversity and 
creativity that have distinguished U.S. contributions to scientific knowledge. 

The advisory panel, workshop participants, reviewers, and other contributors to this study 
were instrumental in defining the key issues and providing a range of perspectives on them. 
OTA thanks them for their commitment of energy and sense of purpose, llieir participation 
does not necessarily represent endorsement of the contents of this report, for which OTA bears 
sole responsibility. 
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Summary and Issues for Congress 



Introduction 

Research provides extraordinary benefits to soci- 
ety through the creation of new knowledge and the 
training of scientists and engineers. The research and 
higher education system in the United States is the 
envy of the world, and has a long history of 
advancing the state of scientific knowledge. This is 
known as •'scientific progress"; . . not the mere 
accumulation of data and information, but rather the 
advancement of our codified understanding of the 
natural universe and of human behavior, social and 
individual."* These advances have addressed such 
goals as enhancing the Nation's public health, 
military security, prestige, educational achievement, 
work force, technological development, environ- 
mental quality, and economic competitiveness. 

lb say only that research contributes to national 
goals, however, simplifies and understates a com- 
plex system. Research is no longer p remote, 
scientist- or engineer-defuied activity resulting in 
new knowledge for society. Perhaps it never was. 
••Deeply held political values of democratic ac- 
countability and public scrutiriy have naturally and 
inevitably impinged on science policy. Demands for 
observable benefits from public investment in sci- 
ence increase. "2 Such demands have led to claims 
that scientific research has a significant and direct 
impact on the economy, and that an investment in 
knowledge is a downpayment on the products and 
processes that fuel U.S. economic growth and 
proJuctivity.' Economists admit, howevcir', that the 
difficulties in measuring the benefits of research 
••...are hard to exaggerate."* The Nation now 
expects that in addition to knowledge, science and 
engineering will contribute to U.S. prestige and 
competitiveness abroad, create new centers of re- 
search excellence on a broad geographic basis, 
continue to provide unparaUelcd opportunities for 




Scientists at th« Res«crch Triangle Institute, NC, 
synthesize chemicals for cancer research. Sdentlflc 
research laKes place In many settings In the United States. 

education and training, and nurture a more diverse 
research work force. 

Thus, the Federal Government funds research to 
achieve more than specific national goals. By doing 
so, it invests in knowledge—and the people who 
produce it — not only for its intrinsic worth (which 
can be considerable), but also for the value knowl- 
edge acquires as it is applied. 

Scientific research is typically split into two 
categories, •'basic" and ••appUed." Basic research 
pursues fundamental concepts and knowledge (theo- 
ries, methods, and findings), while applied research 
focuses on the problems in utilizing these concepts 
and foiins of knowledge. OTA does not generally 

tcll vlty oosoiDf In Hi! of tbeie fieldi. octuniom K(eiic<« u well u tbe lutunl Kieac«« «od et)«iiieeriii«. ' •Rttutch" refera to > creative 

2K«oeth Pwwltl. "-nt PuWlc «od Sclcoce Policy." Science. Technoh,,. A Human Values, vol. 7. .priag 1982 p 13 
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distinguish between these categories in this report, 
because policymakers, especially Congress, make 
very few decisions in which the two are separate. In 
particular, research agency program managers rarely 
allocate monies on the basis of a project's basic or 
applied classification, and divisions of research 
funding into these categories are often unreliable.^ 

This Report and Its Origins 

In December 1989, the House Committee on 
Science, Space, and Technology requested that OTA 
assist it in understanding the state of the federally 
funded research system— its goals, research choices, 
policies, and outcomes — and the challenges that it 
will face in the 1990s. By requesting a study of the 
state of the Nation's research system and of alterna- 
tive approaches the Federal Government could take 
in funding research, the Committee sought informa- 
tion on the nature and distribution of research 
funding and decisionmaking. Direct congressional 
involvement in research decisionmaking is growing, 
and annual agency appropriations seem more closely 
tied to specific goals — and tough choices among 
them— than ever before.^ As one member put it: 

... the payoffs for the Nation are so great that 
increased investments in science and technology are 
only pmdcnt. However, even if we could double the 



science budget twnonow, we would not escape the 
need to establish priorities — '' 

The Federal Government has sustained an illustri- 
ous history of support tor research. Underlying tiiis 
relationship between government and the scientific 
community was a social contract or "trusteeship/* 
developed after the scientific breakthroughs spurred 
by World War n, that delegated much judgment on 
Federal research choices to scientific experts.' 
Perhaps the epitome of the trusteeship was the 
research grant, which created a new relationship 
between the Federal Government and the research 
performer, especially the principal investigator in 
universities.' This social contract implied that in 
return for the privilege of receiving Federal support, 
the researcher was obligated to produce and share 
knowledge freely to benefit— in mostly unspecified 
and long-term ways— the public good.*° 

Since the 1960s, Federal funding for research 
(both basic and applied) has increased from roughly 
$8 billion in 1960 (1990 dollars) to over $21 billion 
in 1990 (see figure 1.) Funding increased quickly in 
the early 1960s during the "golden years" for 
research, after the launch of the Sputnik satellite, the 
escalation of the Cold War, and the Presidential 
commitment to land men on the Moon. Once these 
challenges had been met, research fimding decreased 
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Figure 1-F©d«rally Funded Research (Basic and 
Applied): Fiscal Years 1960-90 
(In billions of 1982 dollars) 
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Slightly and leveled off from the late 19r>0s until the 
nud. 1970s. From 1975 onward, however, Federal 
research funding again increased, due in large part to 
the expansion in health and life sciences research " " 
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perhaps by as much as 60 percent from 1977 to 1987 
(see figure 2). '2 More generaUy, from 1980 to 1988 
scientists and engineers in the work force grew by ra' 
average of 7.8 percent per year, lour times the annual 
rate for total employinem.i^ Not surprisingly, the 
competition for research funds among these scien- 
tists and engineers also intensified. By the late 

(WuhStIl'.D?i^tF^S2S2£.^^^ m«orkal Tablet: Fiscal Ytar, 1955-1990 
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Along with this increase in research funding, the 
number of academic researchers grew steadily, 
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19808, researchers supported by the Federal Govern- 
ment had become increasingly restive over funding. 
Tbday, many say that their lives as researchers have 
become more stressful and laden with the paperwork 
of proposal applications and accountability for 
awarded funds, inhibiting the creativity and joy of 
the research process. They cite the declining 
fraction of meritorious proposals that are funded, 
new investigators lacking the support to set up 
independent research groups, and the fear that U.S. 
students will turn their careers away from academic 
science and engineering." 

Tbday, because the scientific community has the 
capability to undertake far more research than the 
Federal Government supports, policymakers and 
sponsors of research must continuously choose 
between competing "goods." (The tensions under- 
lying these choices are summarized in table 1.) 
Controversies over the support of younger scientists 
and established researchers, "have" and "have- 
not" institutions, and tradeoffs among fields are all 
manifestations of the consequences of choices 
perceived by various segments of the "scientific 
community."**' Scientific community, as used here, 
refers to a political entity. Like other sectors, science 
contributes to national goals and competes for 
Federal resources. At a more practical level, the 
scientific community invoked by Congress and the 
Presidential Science Advisor refers to a heterogene- 
ity of professional associations, lobby activities, and 
actual research performers. (These disciplinary or 
subject-specific divisions and interest groups more 
accurately correspond to what OTA calls "research 
communities.") 



Additional funding for science and engineering 
research would certainly be a good investment of 
Federal resources. There is much that could be done, 
and many willing and able people and institutions to 
do it. The focus of this report, however, is not on the 
level of investment, but on the "Federal research 
system." As the sum of the research programs and 
efforts that involve the support of the Federal 
Government, the "system" is best characterized as 
the conglomeration of many separate systems, each 
with constituencies inside and outside of science." 
How these participants compete, cooperate, and 
interact in processes of Federal decisionmaking 
determines which research is funded by the agencies 
and performed by scientists and engineers. 

If large increases in the budget were to material- 
ize, it would not necessarily relieve system stresses 
for long. Additional research funding would cer- 
tainly allow the pursuit of more scientific opportuni- 
ties and yield fruitftil gains, but it would also enlarge 
the system and increase the number of deserving 
competitors for Federal support. To is, such stresses 
must be addressed with other pc??.oies. In the short 
term, the government faces a rising budget deficit. 
Congress has set targets to reduce the deficit and 
eventually to balance the budget.*^ In this fiscal 
climate, the research system may not be able to 
maintain the growth in Federal funding of research 
that it experienced in the 1980s. Regardless of 
funding levels, however, issues of management, 
funding, and personnel remain. 

Given the extraordinary strength of the U.S. 
research system and the character of scientific 
research, there will always be more opportunities 
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i^AM one ooUtical scientist writea: "... bec«ue the Federal RAD system U compri*sd of so many ind ep enden t acton, each of whom taid to view 
scieo^i^ SS^^Suvely m^perspectlve. d>e F^RAD system proceed, virtually Pl^l"* •»? ^^n^ n 
JiSnT!! Sg slowly and incrcmeottlly in several dlrectiooa at o«e. with conatanUy changing boondaries and shape." Joseph 0. 
Morooe, "Fedend RAD Structore: The Need for Change," Tht Bridge, vol. 19, «iU 1989, p. «. 

>»TTiB debt held by -he Federal Oovcmmcitf reo«ttitly topped $3. 1 trilUoiL and paymenu oo the debt exceeded $255 billion in fiscal )«« 1990- 
fimiwSeSeJtedtotSerignlfto^ 
SSJI^e^uSToftiKBodSEnfJ^ 
17. 1990. pp. 2793-2797. 
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Table 1— Ttntlont In the Ft^eral RetMroh SyMem 



C«nlrallzatlon of Federo! research planning 
Conosnirated exoetience 

"Marksf ' forces to determine the shape of 

the system 
Continuity In funding of senior Investigators 
Peer revtew-based allocation 



Set-aside programs 



Conservatism in funding alocatlon 
Perception of a "total research txidget" 
Doliars for faculties or training 
Urga-soele, nwitlyear, oapital^ntenslve. 

high-oost per-invtstlgator initiatives 
Training more researchers and creating 

more competition for furtds 
Emulating mentors' career paths 
Relying on Nstorlo methods to tXiild the 

research work force 



SOURCE: OHiM of Tiolinology AaMwrntnt. 1991 . 



< — » Piurallstio, decentralized agencies 

Regional ond institutional development (to 

enlarge capadty) 
Poiltloal intervention (targeted t>y goal, 

agency, program, institution) 
Provlsicns for young kwestlgatori 
Other funding decision mectwnlsms (agency 
manager disaetion, congressional ear- 
marking) 

Malnstreamlngcrtterla Inaddltlonto scientific 
merit (i.g., raoe/ethnidty, gender, princi- 
pal Investigator age, geographk: region) 
Risk-taking 

Reality of disaggregated funding decistons 
Dollars for research projects 
individual investigator and small-team, 1-S 

year projects 
Training fewer researchers and easing com- 
petition for funds 
Encouraging a diversity of career paths 
Broadening the participation of traditionally 
underrepresented groups 



< — V 



< — > 



< — > 
< — > 



than can be funded, more researchers competing 
than can be sustained, and more institutions 
seeking to expand than the prime sponsor-^he 
Federal Government— can fund. The objective, 
then, is to ensure that the best research continues 
to be funded, that a full portfolio of research is 
maintained, and that there is a sufficient research 
work force of the highest caliber to do the job. 
This report is designed to support Congress in 
achieving these goals. 

Trends in Federa' Research Funding 

The research system has shown itself to be 
renutfkably robust over at least the last 30 years, and 
it has done well with the resources it has receive/i. To 
develop multiple perspectives on the system, Fed- 
eral funding can be examined by agency, broad field, 
ano cii'.egory of recipient. 

Figure 3 displays Federal funding trends for the 
six largest research agencies.'" Since 1973, tlie 
Department of Health and Human Services (HHS, 
largely through the National Institutes of Health — 
1J!H) has supported more research than any other 
Federa] research agency. In fiscal year 1989, HHS 



supplied nearly twice the research funds of the next 
largest research agency, the Department of Defense 
(DOD). HHS and DOD were followed by the 
National Aeronautics and Space Administration 
(NASA), the Department of Energy (DOE), the 
National Science Foundation (NSF), and the Depart- 
ment of Agriculftire (USDA).20 

Reflecting the division of research funds by 
agency and broad field, a 20-year time series is 
shown in figure 4. Life sciences continues its steady 
growth relative to other broad fields. In fiscal year 
1990, life sciences dominated Federal funding at 
$8.9 billion (in 1990 dollars). Engineering was 
funded at slightly less than one-half the level of 
support given to the life sciences ($4.4 billion), as 
were the physical sciences (roughly $4 billion). 
Environmental and mathematics/computer sciences 
were funded at $2.1 and $0.7 billion respectively, 
and the social sciences together gathered $0.6 
billion. 

Tbrning to research performance, universities and 
colleges in the aggregate are the largest recipients of 
federally funded research (basic and applied, see 



"Coogreuis nwMt uuemted la comjaring reiMreh expeodltiues to oiber eten»oti of ibc PedcnJ budget. THui. > den«tor Out rtpreieou «p«adituret 

for coi«reitk>n.l policy walyui. Given the probieou with re««fcb-fpectnc defl«tori and the advutige of . genend-ONP denJor tTcoiSrt 
STS 2TSc™Sl oS*!^) «loU*r graph, ind uWei in this report were calculated with the ONP Implicit Price DcfUtor for 1982 doUar. 

M»!^T!^iS^x2l^ "L"***' •«encie« would be changed If researeh and development or basic r«c«ch were used to rank them. TTie renaiiung 
igeodcs. not included io the lop ii». together fund less than 5 pcrant of the research supported b> Federal Goveranicut. fcnauung 
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Figure 9— Federally Funded! Research In the Major 
Reeearch Agencies: Fiscal Yeara 1960-90 
(In billions of 1962 dollars) 
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HHS 
DOE 



1970 1976 1960 1965 



1990 



OOD 
NSF 



NASA 
USDA 



KEY: MM&-U.S. D«partm«nt of HMtth and Human S«fvto9a; OOD-U.S. 
Dapartmant of Dafanaa; NASAaNatlonal Aaroraullca and Spaca 
Admlnlatratlon; OOE-U.S. Dapartmanl of Enargy; NSF-Natlonal 
Sdanoa Foundation; USDA>-U.8. Dapartmant of Agrlcullura. 
NOTE: Raaaarchlndudaa both basic and applM. Ftguraawaraoonvarlad 
tooonatant 10e2dollaraui4n9 thaGNP ImpllciS li^rloaDaflatoi'. 1990 
f Iguraa ara aatln>ataa. 
SOURCE: Nalional Sdanoa Foundation. FMm/ Fundb hf Rasaanc/f and 
DMhpmmt. D^fmlM HMoflotii TaMa; FI$<mI Vaa/s f 055- 
fMOiWaahlngton, DC: 1900). tabia A; and National Sdanoa 
Foundation, $0f4»t0dDtitMOnF0d0miFund9forRm0MrchMnd 
D^vwiopnmjUnwcMiYwM 1989, 1990 Mnd 1991 (Washington. 
DC: Oacambar 1990), tab<aa 4 and 5. 

figure 5). From 1969 to 1990, Federal funding for 
research at universities and colleges grew from over 
$4 billion to nearly $8 billion (in constant 1990 
dollars). In 1990, performance of research by 
industry (at over $3 billion) and the Federal labora- 
tories (at over $6 billion) are funded at lower levels. 
For basic research alone (not shown), universities 
and colleges are even more clearly the dominant 
research performer at over $5 billion when com- 
pared with Federal laboratories, the next largest 
basic research performer^ at slightly over $2 billion. 



Figure 4--Federally Funded Research by Broad Field: 
Fiscal Yeara 1960-90 (In billions of 1982 dollars) 
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LIfa aolanoaa 
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-B- Envlronmantal 
aolanoaa 

Math/computai 
aolanoaa 



Englnaarlng 

Social 
aolanoaa 

Phyaloal aolanoaa 



NOTE: Raaaaroh Indudaa both bailc and appllad. RakH not inoludad in 
thia figura ootlacttvaty Moountad for $1 .1 billion (4.9 pm:^\) of aN 
fadaraXy fundad raaaarch In 1990. Flguraa wara oonvartad to 
constant 1982 dolars using tha QNP tmpildt Prioa Daftator. 1990 
flguraa ara astlmataa. 
SOURCE: National Sdanoa Foundatloni M^tbI Funds hr R^ss^roh •nd 
Ch^/0hpfn4nt, DMsihd HtsMomi TMbim: Fkcml Vaars 1955- 
1990 (Washington. DC: 1990). tabia 25; and National Sdanoa 
Foundation^ $0hct0dD0tM on F^chtmi Fundi for Rm^arch Mnd 
D%vhpm%nt:n9o^yw9 1989, 1990 Mnd 1991 (Washington, 
DC: Oacambar 1990). tabta 1. 



The diatribution of Federal research and develop- 
ment (R&D) funds has long been a contentious 
issue— both in Congress and in the scientific com- 
munity. As shown in figure 6, if these funds are 
aggregated by the State of the recipient institution or 
laboratory, tlicn five States received 53 percent of 
the R&D funds in fiscal year 1990 (California, 
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FloureS—Fedorally Funded Research by Performer: 
Fiscal Years 1969-90 (in billions of 1962 dollars) 




1969 1974 1979 1964 



~*- Fed«ral 
Qovernment 

-B- Nonprofits 



-t- Industry 
FFRDCs 



1969 



Universities 
and colleges 



-0~ Other 



KEY: PFRDCt Indudt all Ftcteralty Fundwl R«March and Development 
V enters that are not administered by the Federal Government. Other 
indudes Federal (unda distributed to State and local oovernmentB 
and foreign performers. 

NOTE: Research Includes bo<h basic and applied. Figures were cxjnverled 
to constant 1 082 dollars uslno the QNP tmpHcli Price Deflator 1990 
(iBuras are est Im a tea. 

SOURCE: National Sdence Found/ 'on, FeeHera/ Funds (orRasMrch and 
D*V9hpmwnt, Oefa/M Hitlotiatl Tabim: Fitcal Y»ar$ 1955- 
(Washington, DC: IMO), table 17; and National Sdencs 
Foundation, SaltoladData on Ftdtral Fundi farRtaarch and 
Davahpniant:FlMcat years 1089. 1990 and 1091 (Washington. 
DC: December 1090), taUe 1 . 



Figure 6— Federal R&D Obligations by State (1985) 
and at Universities and Colleges (1989) 

Cumulative distribution of Federal RAD 
obligations by State: 1985 
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Cumulative distribution of Federal R&D 
expenditures at universities and 
colleges: 1989 
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SOURCE. National Science Foundation, Qaographic Pattarns: RiDIn tho 
^rH^ ^l"^' ''•f^'' NSF 90-316 (Washington, DC: 
1990), table B-5, and National Science Foundation, Setocfed 
Data on Aatdamk: Sdanca/Enginaaring RAD Expandlturas 
Fiscal Yaar 1909, NSF 90-321 (Washington, DC: October 
1990), table B-36 end CASPAR database 



Maryland, Massachusetts, New York, and Vir- 
ginia).^' (Research institutions are also not ran- 
domly dispersed across America; rather, they are 
concentrated on the two coasts and the upper 



iiiidwest.) ; .t the other end of the distribution, 15 
States together receive<l less than 2 percent of the 
funds. At the institutional level, 10 universities 
receive 25 percent of the Federal research funding, 

Na»loiul Science Foundatloo and thc National ^lituie, of H^,h^?n„^^^^ '° over one-tudf of the proposal j to the 

Ihe swards. See U.S. Ge««l f^uZomcXS^Zt^ "2^"^"" P™P«"' '^'1 *on over W^percentTf 

DC: Febnury 1987). p. 43. iS^eVe? JS^^^ ^'"^ f^'"v.r,/„« (wLnglo^ 

J* toul p«puJ«ion of a SUte and theTmub^^rS^^ 

be «en as "unfair." Also i<c Wllli«n C. Boesman ..k1 ciristiSf Sh^3o« .^iSS^^ competiuvc process, the outcomes may 

Dcvelopmea. l^nds." CHS Hcponfor Con^rrss ,^^To^:T^;i^^':a^ aSh tr:'Spr^5%8t''^ 
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and only 30 universities account for 50 percent. 
Funding is concentrated in 100 research universities 
in 38 States. This reflects their importance to the 
Nation's research enterprise. 

These data on the distribution of resources bear a 
critical message: research c^abilities— institutions 
and people— take time to grow. It is not simply a 
matter of "they who have, get." The reputation, 
talent, and infrastructure of research universities 
attract researchers and graduate students.^^ Some 
universities become assets not only in the production 
of fundamental knowledge, but also in bridging 
science and technology to other goals such as State 
and regional economic development. 

Federally Funded Research in the 1990s 

Snapshots of federally funded research, compar- 
ing fiscal years 1980 and 1 991, are provided in table 
2. Research is a small portion of the total Federal 
budget. Although the distribution of research funds 
by agency sponsor, category of performer, and 
stratum of academic institution has hardly changed 
during this period, tiie activity has never been in 
greater demand. 

However, questions such as "Does tiie Nation 
need more science?" and "How much research 
should the Federal Government support?" have no 
ready answers. Measures of distress and conflicts 
over resource allocation within the scientific com- 
munity do not address whether the Nation needs 
more science. Other problems in the Federal re- 
search system do not derive from, but are exacer- 
bated by, such stress. They include sparse participa- 
tion by women and eUinic minorities in science, 
indications that other nations fxe better able to 
capitalize on tiie results of U.S. research tiian 
American industry, and management problems tiiat 
have plagued many Federal research agencies. Only 
some of these problems can be addressed solely by 
tiie Federal Government, and long-term solutions 
may not be found in adjusting Federal funding 
levels. Ratiier, tiiey reflect problems in Uie orgamza- 



tion and management of research and competing 
values within tiie scientific commuxuty.^ 

"How much is enough' ' depends on tiie goals of 
tiie research system (see box A). The system by 
definition takes on new goals, each of which can be 
evaluated. But in tiie aggregate how tiiese goals are 
assimilated — by add-on or substitution — is not eas- 
ily predicted. Tne challenge is not to determine what 
fraction of tiie Federal budget would constitute 
appropriate funding for scientific research. Rather, 
OTA finds that under almost any plausible 
scenario for the level of research funding in the 
1990s, there are issues of planning, management, 
and progress toward national goals to address. 

Because tiie reach of science is now great, 
decisions about tiie funding of research are inter- 
twined witii many Federal activities. Congress and 
tiie executive branch, which make tiiese decisions in 
our form of government, will continue to wrestie 
witii scientific and otiier national priorities, espe- 
cially tiiose tiiat help prepare for tomorrow's sci- 
ence — renewing human resources thioughout tiie 
educational ^npcline and building regional and 
institutional capacity. History cautions against tiie 
expectation tiiat tiie scientific conmiunity will set 
priorities across fields and research areas. Congress 
must instead weigh tiie arguments made witiiin each 
area against desired national outcomes. 

In tiie 1990s, tiie Federal research system will face 
many challenges. OTA has organized tiiem here 
under four interrelated issues: 1 ) setting priorities for 
tiie support of research; 2) understanding research 
expenditures; 3) adapting education and hiiman 
resources to meet tiie changing needs of tiie research 
work force; and 4) refining data collection, analysis, 
and interpretation to improve Federal decisionmak- 
ing. (For a summary of issues and possible congres- 
sional responses, sec table 3.) lb craft public 
policies for guiding tiie system, each issue is 
outiined in tiie following discussion. 



ainslituUoni like the faculty re»e«rchcn employed by them, tccumulttc "KlvuiUtc." Among the miny ficton Oat Influence Federal rac;. h 
fu^v^Zt!^iZ^ur.^ioX ptn of . c^cle of cUwiity give. lnve«lg..or. « odfe in ««p«idon for^ SST^KL'ST^^t 
ttul Mtxcngiben the in.tlVution u . productive re«*n=h perfonner . which ^^^^.'Tf^^^^'^'J^^' T*f TJJS^ ^iJSS 
in Science. U: Cumulitivc Advantage and the SymboUun of InteUectual Property,'" Mi, voL 79. No. 299. 1988. pp. 60M23. 

»See JoihuaUdeTbefg.''Doei Scientific Progrew Come From ProjccttorPeoplcrCur««C^^^^^^^^ 
OTA crncSuci on Federal, especially agency penpectivc4 on reiearch. Pcrforaier (leiearcher and in»t' .ational) response, to cbangesto Federal 
^"Si XoS™ werltluS toUden uKlding of the Federal role vU-a-vis acaJ^^c 

for research performance and most dau are collected of> universities. However. naUonal Uboratorie. and industry play targeted loles and figure 

proiuinenily lu research funding decisions. 
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Table 2-Federally Funded Research In the 1980s and 1990s (in percent} 



R4D as percent of total Federal budget 

Total researdi as percent of Federal R4D 

Basic research as percent of Federal R&D 

Basic research as percent of total Federal budget 



Rscal year 1980 



5.0 
38.9 
15.7 

0.8 



Percent of total (basic) research funds distributed, by agency 



Agency Hsoai year 1980 



HHS/NIH 
DOD 
NASA 
DOE 
NSF 
USDA 
Other 



29/24 (38/35) 
20 (12) 
14(12) 
11 (11) 
8(17) 
6(6) 
7(4) 



Percent of total (basic) research funds, by performer 



Perfornner Rscal year 1980 



Universities 
Federal 
industry 
Nonprofits 
FFRDCs* 



32 (50) 
32 (25) 
18(7) 
6(6) 
11 (11) 



Rscal year 1991 (est.) 

4.7 
36.3 
19.1 

0.9 

Rscal year 1991 (est.) 

34/29 (40/37) 
15(8) 
16 (15) 
12(14) 

9(15) 

5(5) 
10(4) 

Rscal y ear 1991 (est.) 

36747) 
30 (23) 
15(9) 
8(9) 
11 (12) 



Percent distribution of Federal R4D funds at academic institutions" 



Ranking Rscal ye ar 1980 Rscal year 1988 



Top 10 
Top 20 
Top 50 
Top 100 



25 
40 
68 
84 



25 
39 
65 
85 



Issues and Options for Congress 



ISSUE 1: Setting Priorities in the 
Support of Research 

Summary 

Priorities are set throughout the Federal 
Government at many levels. At the highest 
level, research priorities are compared to 
nonscience and nonengineering needs. At the 
next level, priorities are set across research 
fields, such as biomedicine and mathematics. 
Within fields, agency programs reflect re- 
search opportunities in subfields and relevance 
to national needs. FinaUy, research projects are 
compared, ranked, and awarded Federal fiinds. 

ERIC 



Although priority setting occurs throughout 
the Federal Government, it falls short in three 
ways. First, criteria used in selecting various 
areas of research and megaprojects are not 
made explicit and vary widely from area, to 
area. This is particularly true, and particularly 
a problem, at the highest levels of priority 
setting, e.g., in the President's budget and the 
congressional decision process. Second, there 
is currently no mechanism for evaluating the 
total research portfolio of the Federal Govern- 
ment in terms of progress toward many na- 
tional objectives, although recent efforts by the 
Office of Science and Technology Policy have 
lead to some cross-agency planning, budget- 
ing, and evaluation. Third, the principal criteria 
for selection, scientific merit and mission 
relevance, are in practice coarse filters. Con 
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Box A— How Much is Enough? 

''How much is enough money for research?" is a question that can only be asked if it is clear what scimtiftc 
and engineering research in the United States is attempting to accomplish: research for what? 

1 . Is the primary goal of the Federal research system to fund the projects of all deserving investigators of 
natural and social phenomena? 

If so, then there will always be a call for more money, because research opportunities will always outstrip 
the capacity to pursue them. 

2. Is it to educate the research work force, or the larger science and engineering work force, needed to supply 
the U.S. economy with skilled labor? 

If so, then support levels can be gauged by the need for more technically skilled workers. Preparing students 
throughout the educational pipeline will assure an adequate supply and diversity of talent. 

3. Is it to promote economic activity and build research capacity throughout the United States economy by 
supplying new ideas for industry and other entrepreneurial interests? 

If so. then the support should be targeted in line with our efforts to pursue applied research, development, 
and teclmology transfer. 

4. Is it all of the above and other goals besides? 

If so, then some combination of these needs must be considered in allocating Federal support. 

Indicators of stress and competition in ti^e research system do not address the question of whether science needs 
more funding to do more science. Rather, thoy speak to the organization and processes of science and to the 
competitive foundation on which the system is bui^t and that sustains its vigor. 

Education, economic activity, and other national goals have long been confronted by Congress and the 
executive branch. Although the relative importance of these needs varies over time with new developments and 
crises, their absolute importance has not been set. Thus, allocating resources to these needs has always been a 
tradeoff, within a limited budget, against other national goals and the programs that embody them. 

Because of its intrinsic merit and in^portancc to the Nation, research has consistently been awarded funding 
increases. But these do not compare to what some claim would be an appropriate level of funding for research to 
pursue a full agenda of opportunities. Deciding if the Nation is pursuing enough research opportunities or if the 
Nation needs more science is thus a complicated question, which requires that other decisions about the nature of 
the research system and its goals be senled first. Table A reports the costs of some potential science initiatives as 
estimated in tlie late 1980s. 



Tablo A—Sample Requests From the Research Community for Increased Funding 

Reld or agency Report or Intlatlve Addittonat fufxls requested* 

NSF Initiative to douWe the NSF budget $2.1 Wlllon 

NASA space sclenco Towards a New Era In Space: ReeUlgnlng U.S. Policies to New Over $1 billion 

Realities'' 

Neuroscience 1Q90s Decade of the BraJn Initiative^ Over $1 billion 

USDA researcti grants Investing In Research"^ $0.5 billion 

Behavioral and social sciences .... The Behavioral and Soda) Sciences: Achievements and $0.26 billion 

Opportunities* 

Mathematical sciences Renewing U.S. Matheouitlcs* $0.12 Ulilon 

All academic research Science: The End of the Frontler?q Over $10 billion 

KEY: NSF»NatlonAi Sdsnc^ Foundation: NASA-Natiocal Atronaulk» and Spaoo Admlnlttratton; USDA-U.8. Dtpartm^ of Aghouttur^ 
AAdjuftod to 1900 dollars using tha 1&d2 QNP Implicit Prkm Daftator. 

^National Acadamy of Sdancas/Natlonal Acadamy of Enginaarlng, Committaa on Spaoa Policy. "Towards a Naw Era In Space: Realigning U.S. 

Poltciaa to New Rtalitias/' ^aca Poiicy, vol. 5, August 1009. pp. 237*25S. 
c** Brain Dscftda' Naurosdantlsts Coun Suppon.** TTm Soi0nm vol. 4, No. 21, Oct. 29. 1990. p. a. 
^National Research Council. Investing In Rwarch (Washington. DC: National Acadamy Press. 1969). 

^National R^aardi Council, 77)s BahMvicfl Mnd $o(M Sdmyo^M: AchlavmrmiU Mnd 0pp0ftuM4i (Washington. DC: National Academy Prass, 

1906). 

^Naticnal Rsssarch Council, Renewing U.S. MMth^matiCM: A PiMn for th0 1990$ (Washlngtoni 00: National Academy PrmM, 1990). 
Q$ci»nc0: Tho End of th$ Fmnilf? a rsport from Uon M . Lsderman. Prssldant-Elsct to the Board of Dlractors of the Anisrtoan Association for the 
Advancamsnt of Sdanca (Washington. DC: American Association for the Advancement of Sdanca, Jan. 31* 1991). 

SOURCE: Office of Technoiogy Ass«ssment. 1001 . 
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Table 3— Summary of Issues and Possible Congressional Responses 

Issue 



Setting priorities for research 



Possible congressional responses 



Coping with changing 
expenditures for 
research 



Adapting education and 
human resou'^ccs to meet 
futur* needs 



Refining data collection and 
analysis to improve re- 
search decisionmaking 



SOURCE: 011ic« of Technology AssMSment. 1991 



Hearings on cfosscutting priorities and congressional designation of 
a body of the Federal Government to evaluate priority setting. 

Application of criteria to: a) promote education and human resources. 
fajbuii'-J regional andinstitutionalcapadtyinmerlt-bcsedresearch 
decisionmaking, and c) balance little science and megaproject 
Initiatives. 

Oversight of agency research programs that focuses on strategies to 
fulfill the above criteria, and on responses to priority setting. 

Encouragenient of greater cost-accountability by the research agen- 
cies and research perforniers (espedally for indirect costs, 
nfwgaprojects, and other multlyear Initiatives). 

Allowance for the agencies to pursue direct cost containment 
nfieasures for specific items of research budgets and to evaluate 
the eff€K:tlveness of each nieasure. 

Programs that focus investnient on the educational pipeline at the 

K-i2 ard undergraduate levels. 
Attention tc dl/erslty in the human resource base for research, 
especially to the contributions of underpartidpating groups. 
Incentives for adapting agency programs and proposal requirements 
to a changing model of research (where teams are larger, more 
spedaiized, and share recn'irch equipment and fadlities). 
Funding to: a) augment within agency data collection and analysis on 
the Federal research 8y5(em, and b) increase use of research 
program evaluation at the research agendes. 
Encouragement of data presentation and interpretation for use in 
policymaking, e.g., employing indicators and other techniques 
that measure outcomes and progress toward stated objectives. 



cerns for developing human resources and 
building regional and institutional capacity 
must also be considered; these criteria 
strengthen ftiture research capability. While 
not every project or agency will factor these 
criteria equally, the total Federal research 
portfolio must address these concerns. 

Ptiority-setting mechanisms that cut across 
research fields and agencies, and that make 
selection criteria more transparent, must be 
strengthened in both Congress and the execu- 
tive branch. Congressional oversight must 
evaluate the total Federal research portfolio 
based on national objectives, research goals, 
and agency missions. In the executive branch, 
Congress should insist, at a minimum^ on 
iterative planning that results in: a) setting 
priorities among research goals, and b) ai^ly- 
ing (after scientific merit and program rele« 
yance) other criteria to research decisionmak- 
ing that reflect planning for the future. In 



addition, since megaproject costs affect the 
ability of other disciplines to start new, large 
projects, megaprojects are candidates for 
ao*scutting priority setting. 

Discussion 

Priority setting can help to allocate Federal 
resources both ^A^hen they are plentiful, as they were 
in the 1960s, and when they are scarce, as expected 
through the early 1990s.^ Governance requires that 
choices be made to increase the benefits and 
decrease tlie risks to the Nation. Priority setting 
occurs throughout the Federal Government at many 
levels. At the highest level, research priorities are 
compared to nonscience and nonengineering needs. 
Ai the next level, priorities are set across research 
fields, such as biomedicine and mathematics. Within 
fields, agency research programs reflect research 
opportunities in subfields and relevance to national 
neuls. Finally, research projects arc compared, 
ranked, and awarded Federal funds. 



(Pumfc31lT82r^^ n Pn?r?"^^^^^^^ ^^^r? ''^''""'^ '^^^'^^ Tcclmology Policy. Organizatior, and Prioritic Act of 1976 
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Uncterground nuclear test craters dot Yucca Rat at the Nevada Test Site (NTS). In addition to nudear testing, researchers at NTS 
explore other scientific phonorrona, such as geologic and seismic problems. 



Toward More Explicit Priority Setting 

Tliere are three problems v "(li priority setting as 
it is currently practiced in the Federiil Government. 
First, criteria used in selecting various areas of 
research and megaprojects are not made explicit, and 
vary widely from area to area. This is particularly 
true, and particuhu-ly a problem, at the highest levels 
of priority setting — e.g., in the President's budget 
and the congressional decision process. The best 
developed p.nority-setting mechanisms are within 
the research agencies and at the agency program 
level. 

Seconds there is currently no mechanism for 
evaluating the total research portfolio of the Federal 
Govermnent in terms of progress toward many 
national objectives. Research priorities must be 
considered across the Federal research system, and 
in particular, across the Federal agencies. What the 
Federal Government values more or less in research 
can be inferred in part from the Federal budget, but 



there is no **research budget/' F'ederal support is 
dis xibuted across many executive agencies, and falls 
unv ;r the jurisdiction of a number of congressional 
committees and subcominittees (see table 4), There- 
fore, once allocations have been made to agencies 
(by the Office of Management and Budget — 0MB) 
or to appropriations subcommittees (by full appro- 
priations committees), decisions are made indepen- 
dently within narrow components of what is after- 
the-fact called the research budget. T\m hampers the 
implementation of crosscutting comparisons by 
Congress. 

During the 1980s, 0MB was a surrogate for a 
crosscutting agent, with Congress adding its own 
priorities through budget negotiations.^^ Recent 
efforts by the Office of Science and Technology 
Policy (OSTP) have lead to cross-agency planning, 
budgeting, and evaluation in certain rcseaich and 
education arc us. President Bush has investe<} more 
power in OSTP to participate witli 0MB in delibera- 
tions over rcseaich spending, especially in targeted 
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^M or an overview, sec Hh/uibcth BiUdwm and Clu islophcr T. Hill. ' 'The Budgci l^occss and Laigc- Scale SciciKc FuxKlii\^%" CHS Review, Pcbiiiary 
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Table 4-CongM88lonal Authorizing CommlttMs and Appropriations 
Subcommltlaai With Significant Ugislativa Authority Ov^ RiD 

Juftgdlctlone of authorizing oommlttees: ' Agwcy " — ' ^' 

House: ' ~~ — ^ ^— — . 

Agflculture 

Armed Services DOD DOE 

inK^SSirASrs\\"::::;:;;;:7: ••••••••••• goE:ADAMHA.NiH.cDc.DOT 

?Srv*^?r;d\"rln;SS!Z"^ ••••••••• NASA^ NSF DOE. EPA. NOAA. DOT. NIST. DOI 

Merchant Marine and Rsheries uQnA wnl a hot 

Veterans' Affairs ^If 

Foreign Affairs aj.d. 

Senate: 

Agriculture, Nutrition, and Forestry ..q-i. 

Armed Sorvices hop 

Commerce, Science, and Transpatatlon nSf fSlsA dot woaa wiqt 

Energy and Natural Resources n&f, nasa. DOT. NOAA. NIST 

Ubor and Human Resources '..^y^y^y^[y^]/^'.'.'.'.'.'''' JSi adamha. cnc h^p 

Envlrcflnrwnt and PuWIc Wtorits cba ^"^'^"^ "SF 

Veterans' Affairs 5a 

Foreign Relations 



Jurisdict ions of authorizing oommlttees: ' ~ Agency " — " 

l^'- Health and Hunran Services. Education ' 

and Related Agencies ^..^ .-iai^ua ^rs^ 

HUD and independent Agencies Kas^ nsp^p2°n.a 

Energy and Witer Development ^^A' 

Interior and Related Agencies , 

Agriculture, RuraJ Development and 

Related Agencies^ 

Commerce. Justice, State, the" judiciary. ^^^^ 

and Related Agencies 

Transportation and Related Agencies KnV^ '^'^ ' 

Foreign Operations r^Z, 

::::::::'.v.v.v.v;;;:;;;:; ::::::::: ftjp 

EWUU.S. Envtronrntrrttl Proltolk^Aww- HuKuT SfS-uL^ J"'"'*' DOT-U.S. D«p«rtm«nt of T™iniport.llon 



SOURCES: 0«^^,T»J«fo^^^^^ 



Presiderdial priority areas such as high-perfomiance 
comp»'.iing, global environmental change, and math- 
•niatics aiid science education.^* Since the Adminis- 
trau.v. u is moving in the direction of more centralized 
and coordinated r-iority setting, it is all the more 
important for Congress to consider priority-setting 
mechanitms as well. 



Third, although scientific merit and mission 
relevance must always be the chief criteria used to 
judge a research area or agency program's potential 
worth, they cannot always be the sole criteria. In 
particular, the application of criteria that augment 
scientific merit—which represent today's judg- 
ments of quality— would help meet tomorrow's 




ocKunli^ prioriUe. too«gt, a» Office of Sdcn2rSK;o^gy pJuS^ S^'- ground n«lc. for dtt^ 

Committee meduoiun we detailed ta the OfHce of iZ^S 2d bS£ wSS '^^f^l^ °" ^°«^««*°«. Ted»ol^ 

during .„ ^.ervlew wiO. Ro^ E. Or.dy, As^iJ:^Ti^:^K^L:^:^ JS^T J^^^^^i" 7^99?^' '"^ 

2i 
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Box B— Criteria for Researcli Decisionmalcing in ^cncy Programs 

Withto agency Kfcaich programs, research pfoposab luve ti^^ 
of expert peer or program manager judgments of ' 'scientific merit' ' and program relevance. Many Federal agencies 
are now fin^lj^ that the introduction of other ejqpUcit criteria is important for research decisionmaking.' 

For t «ampie. the National Science Board (NSB) esuUished the following criteria for die selectioQ of research 
projects 1/ the National Science Foundation (NSF): 1) research perfomier competence, 2) intrinsic merit of the 
research, and 3) utility or relevance of the research. In addition, NSB hichided 4) the **. . . effect of die research on 
the infrestnictuie of science and engineering. This criterion relates to the potemial of the proposed research to 
contribute to better uadentanding or improvement of the quality, distribution, or effectiveness of the Nation's 
scieotiAc and Mginening research, education, and manpower base."^ 

Under this fourth criterion, NSF includes: 

. . . questioas relating to scknttfio, engifleedni, and education penomel, including puticij^adoa of women, 

rainoritiea, and disabled individuals; die distribudon of resources widi i«f»ct to ini 

•titnulatieft ot high quality activitiea in irapoiiaat but underdeveloped fields; mppon of retsiuGh inidatioa for 

inveitigatofs widwut pnvieas Federal reaeaidi Bt9pon as a Rrbd^ bwe^^ 

intoflUieiDURtfv noraadiat to rcioaich ot edticadon in aoDraDriats araai.' 

In short, this critericQ defiOM the baaet for luing other criteria in additioQ to icienUfic merit in miinstttam 
aUocatiooi of nieuch fUttU , and wi^ 
the oootbming need for ''ihelteied competi^ 
(nwgtaui.* 

As acknowledged NSB, although sdeotific merit and 
uMd to Judge a reaekch pragnm or pn^ 
today *t Rteaich pKiiamit theie are many moK 



^OtKmtfkw$. itti^mumm 1990. 

SQiioMI la Kiiioeil IdMM PooDdMioa* Oram far Rttionh and Edneatkm in Sckuct and BHgtnurifit : An Apptt€ation Guidi, HSV 
90-77 (WadOiviDiipOC iUgvit I99(9tpp. 

poo p ti»TlNi><igMriofict <w ( f9 i li awiliaiPiiypoitel^ 
hinaaa iwnrcce ickacc aodcpitaie^^ 



objectives of research investment. Broadly stated, 
there are two such criteria: strengthening education 
and human resources at all stages of study (e.g.. 
increas'ng the diversity and versatility of piutici- 
' ns); and building regional and institutional capac- 
(including economic development by matching 
Federal reseiu'ch support witli funds from State, 
cor|X)ratc, and nonprofit sources). 

Fxlucalion and human resources CTileria would 
weigh research initiatives on their *'pr(xluciion" of 
new re.seaichcrs or technically skilled students. 
Contributions to human resources include increas- 
ing participation in the educational pipeline (through 
degree completion), the research work force, and the 
larger science and engineerijig work force. Regional 
and institutional capacity criteria would weigh 



research initiatives on their contribution to under- 
Piirticipating regions and institutions. Regional and 
institutional capacity are important concerns in all 
Federal funding, and encouraging new institutional 
participants and development of research centers 
strengthens the future capacity and diversity of the 
re.search system. Some agency progrmns already 
incorporate these criteria in project selection (see 
box B). 

Can Congress look to the scientific community 
for guidance on setting priorities? llie short answer 
is "no.*' Congress wishes — perhaps now more than 
ever — that the scientific community could offer 
priorities at a macTO level for Federal funding. 
Science Advisor Bromley and former Science Advi- 
sor Press have stated criteria and categories of 
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review (XHild thus be im ttenube pRKest. 

program relevafwe* tnd ttxMe with exoepcional human leiouroei and/or leieafch infimtractuie potential so 

imUcated The program manager, with or whbout the ad 

the pool Any of several sulMeU might be equallly meritortoi^^ 

the ptocesi. llie result is a program reseaicfa portfolio that ^ 

01A suggests that two broad eritetia oouki be apj^ 
and human tetouroest and building regional and institution 
rated in res^ch proposals? 

• Education and human resources criteria would weigh proposals on their fiituie production of new 
researchers or technically skilled students. Outeome measures would relate to undeignduate education* 
graduate training* and characteristics of new Pbl).i-^ num^ 
study and die research woik fbice» respectivdy. 

Contributions to human resoutcea inehide hicieashv participation in die educat i o n al pipeUne (d^rough 
degree comirfetlon)* di^ researdi woik foree> and t^ 
changing character of the studempopdatidfu 
in science and cqgin^^ 

woffcfofce. . ^--^ . . ; ^■ 

e Regionai and insHmttondt ai^ weigh propoials on their odotrlbution to undeiparticl* 

pating fvgioM and insitokws (Mcote 0Mis«ei wM the eoiianoed raceiucfa competitiveBess 
of ftmdediostitutioiui Stiii* iddi^ 

an enlsrged role in trtdnJng and mii/k^^ lirgilsd aectaiti iMtoes, and fields. 
Regioiud and insiiw^^ aUM^ llih^^ refiecting the interests of 

taxp^yttfc^WUkt thfe^ of research* untmd 

resources could bsi devi^^ oteQ^ the United States*^ 

Pundidg feseaicb to addevil iQ^d^^ dl^iectlvea frill rebudn t pnvQg«fv« of Congress* But decisions that 
addtoniocrow*scritetiittotoday%espe^^ 
the capdiili^ of die Federal r«sestiGh syitcau 



%fe»lW<ri O(titf iM i iii < w ti> iSieiSgaa«liisice ^^ 
ofsdrisvi^tllt 'MffaKOffi** mnmh mm imm traditioaiHy knftoHi locsl nt!Mm\$i ty woarstiag dmlopttcnt of lechntad 
IndttHries sad ioeH leK s whlaft aii iaa s iei Hny alie rtftiaceairfciis to locsl s ihcaHo m l jftaiis sad dtwctly ptevids tichaicsi jobi for 
mldsBiit. See VS. Ooqsmi, Ollios ef TMrnkegf ^nnsmssi, Mttlut Bdmedon fsr Scimee Md Oiglnmintt 01C4*11MBT>52 
(WAiititiftoo, DC: U J. QovmoMt Msciai OMoe» Mwcb 1989)* 



priority that they consider essential for science. 
Each emphasizes the separation of large projects 
requiring new infrastructure from **small science.'* 
Press further distinguishes human resources from 
national crises and extraordinary scientific break- 
throughs, whereas Bromley places national needs 
and international security concerns above all else.-^**^ 

While tlic I^ess and Bromley fonnulations apfx^ar 
to provide frameworks for pnority setting, they do 
not address the problem tliat there are few mecha- 
nisms for, and no tradition of, ranking resefU"ch 
topics across fields and subfields of inquiry. In 



addition, priority setting is often resisted by the 
recipients of Federal funding because it orders the 
importance of reseiirch investments, which means 
that some programs do not get funded and some 
groups within the scientific community coinphun of 
lack of support. Consequently, Congress and the 
executive branch have found that the scientific 
community cannot make crosscutting priority deci- 
sions in science. In particular, the traditional mecha- 
nism of peer review is clearly not suited to making 
judgments across scientific fields. Some research 
communities do set priorities within specific re- 
search areas. However, tlie practice is not universal 



^Scc I'rank Press, "'Tlic DUcmnia of the (ioldcn \f^t:' Conf^ressional Rfconi, May 26. 1988. pp LI 738-1! 1740. ajid D Allan Bromley, "Krynotc 
Address." in Saucr, up cii . footnote 1 1, p. 11 (This wus augnienicd by "US. Technology Policy/' issued by the fUctutivc Office of ihc Prcsidciu, 
Office of Science and Technology Policy. Sept. 26. I9<X)) 

^*()nc effect of tlicsc mnk-orders is the seeming aeuUon of separate actnunts. i c . ihiit choices could be made within each category and tlicn across 
categories (Jf toursc, such cfioiccs arc being made by various piirticipants in the rcsciuch system siniuliaricously 
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or widespread.^' Ihcreforei while recognizing the 
preferences of researchers, the Federal Government 
must set priorities at two levels; among scientifically 
meritorious research areas and megaprojects, and 
among agency programs. 

Megaprojects and the Science Base 

Key to the consideration of allocating public 
funds for science and engineering research is the 
simultaneous support of little and big science. Little 
science is the backbone of the scientific enterprise, 
and a diversity of research programs abounds. Not 
surprisingly, many investijfjators and their small 
teams shudder at the thought of organizing Federal 
science funding around a principle other than 
scientific merit — an approach that, in fact, is advo- 
cated, by no one. They fear that setting priorities 
would change the criteria by which research funds 
are awarded. In particular, they seem to hear calls for 
priority setting as calls to direct all of research along 
specified lines, not as a means to assure that balance 
is achieved. For example, one goal would certainly 
be the maintenance of funding for a diverse science 
research base,^^ while other goals would include 
training for scientists and engineers, and supplying 
state-of-the-art equipment. 

The Federal Government also seeks to achieve 
goals at man]^evels. These goals are likely to differ 
between pro^bns that pursue specific objectives 
and those that seek primarily to bolster the science 
base. For instance, the allocation of additional 
.nonies to NUi for AIDS (acquired immunodefi- 
ciency syndrome) research, beginning in the late 
1980s and continuing today, has been a clear 
designauon of an objective as a priority research 
area. In addition, to enhance the science base in 
specific research areas, such as environmental sci* 
ence and high- temperature superconductivity, the 
Bush Administration has increased funding in cer- 



tain fields. These increases, howevrer, seem to be 
dwarfed by the cost of a very few, but visible, 
mega{»ojects. 

Megaprojects are large, *'lumpy,'' and uncertain 
in outcomes and cost. Lumpy refers to the discrete 
nature of a project. Unlike little science projects, 
there can be almost no information yield from a 
meg^roject until some large-scale investment has 
occuxred. Presumably, a successful science mega- 
project provides knowledge that is important h'nd 
unattainable by any other means. Because of the 
large expenr . Aires and long timeframes, many 
science megaprojects are supported by large politi- 
cal constituencies extending beyond the science 
community.^^ Future decisions may center on rank- 
ing science megaprojects, since not all of them may 
be supportable without eroding funding of the 
science base (see figure 7). 

There are few mles for selecting and funding 
science meg^ojects; the process is largely ad hoc. 
From a national perspective, megaprojects stand 
alone in the Federal budget and cannot be subject to 
priority setting within a single agency. Nor can 
megaprojects be readily compared. For example, the 
Superconducting Super Collider (SSC) and the 
Human Genome Project (HGP) are not big science 
in the same sense. One involves construction of one 
large instrument, while the other is a collection of 
smaller projects.^^ 

An issue raised about some megaprojects is their 
contribution to science. For instance, the Space 
Station has little justification on scientific grounds,^^ 
especially when compared with the SSC, the HGP, 
or the Earth Obser\'ing System, which have explicit 
scientific rationales. On purely scientific grounds, 
the benefits that will derive from investing in one 
project are often incommensurable with those that 
would be derived from investing in some other.^ 



^or exjunplct. lee the Nttiooil Research Council, Rtruwini US. Mathematics: A Plan for fht 1990s (Wuhlogtoa, DC: Nttioaii Ac«dcmy Press, 
1 990); and the Nttioiul Research Council Tlu behavioral and Social Sciences: Achievtmenu and Opportunities (Washimrtoo, DC: National Academy 
Preu, 1988). 

'^a piiority hu been pfeaninem since the Federal support of research began. See House Committee on Science and Technology, op. cit. » footnote 

9. 

5iphiJ Kuntz. "Pie in the Sky: Big Science Is Ready for Blastoff/* Congressional Quarterly, Apr. 28. 1990. pp. 1254-1260. 

research supported by the Human Genome Project— HGP— may have sotne scientific beoftfits before the project is complete. Thus. HGP may 
not be big science in the strict sense of the definition outlined above. See Ibm Sboop. "Biology's Moon Shot.* ' Government Executive, February 199:. 
pp. 10-11. 13. 16-17. 

"For an early statement of thU view, see U.S. Congreas. Office of Technology Assessment, Civilian Space Stations and the US. Future in Space. 
OXA-STI-241 (Springfield. VA: National Technical Informatioo Service, Novembw 1984). 

^This Is elaborated in Htfvey Avttch. ' 'Analyzing the Cosu of Fedcnd Research,* * OTA contractor report, August 1990. Also see Sigcl e< al.. 
"Allocating Resources Among AIDS Research Strategies.** Policy Sciences, vol. 23. No. 1. February 1990. pp. 1-23. 
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Figure 7— Cost Scenarios for the Science Base and Select Megaprojects: Fiscal Years 1990-2005 
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NOTE: ThttM ftgurM ars ichtmatic rtprgMntatlona of proj»ctsd oosta for sa^nco projects. In th« figures on tho left. th« sd9nc» basB Is proJoct«d to grow 
at An annual rait of 3 ptroont abov* infiatlon. In th# figuro on !h« right. \otMl Fwd^rai fswch funding \s proJ(»cted to grow 3 p«rc4»nt abov« Inflation. 
Th« oott ««timat«t for th« m»gaproJ#ctt kf bSMd on data from 'Tha Outlook In Congrtaa for 7 Major Big Sclonoa Projects." 77)a Chronfch of Higher 
Education, Sap<. 12. 1990. p. A28. and Ganaviava J. Knazo. Congfasalwiai Rasaarch Sarvica. Scianca Policy Rtiaaarch Division. "Science 
Magaprojactt: Status and FurxJIng, February 1901," unputtlshad decuman!. Feb. 21. 1991. 

SOURCE: Office of Technology Aasesamant. 1991. 
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Fhoto ctediU Naflonal Afonaullcs and $p«c« Administration 

The Hubble Space Telescope (HST) is shown, still In the grasp of Space Shuttle Discovery, with only one of two solar panels 
extended Earth Is some 332 nautical-miles away, HST is an example of a large scientific 

mission at NASA. 



However, because the problem of selecting iimong 
science inegaprojects has most in common with llie 
selection of complex capital projects, timeliness 
(why do it now rather than later?) and scientific and 
social merit must all be considered, as well as 
economic and labor benefits. At present, for exam- 
ple, the Space Station has considerable momentum 
as an economic and social project. 

Other measures to evaluate and, if necessary, 
compare megaprojects include the number and 
diversity of researchers tliat can be supported, the 
scientific and technological value of information 
likely to be derived (i.e., the impact of the mega- 
project on the reseaich community), and the ultimate 
utility of the new equipment and/or facility. For 
instance, if one project will support only a few 



researchers, while a second of simihir cost and 
scientific merit will support a larger number of 
researchers, then perhaps the seco..:'- ':hould be 
favored. One might also expect preference foi 
megaprojects that can be cost-shared internationally 
over those that cannot be. (Issues of cost.s in 
megaprojects are discussed below.) 

Once the context for priority setting is examined, 
choices take on another dimension. What do U.S. 
society and tb -"^ Federal Government expect for their 
research investment? V»hat does the scientific com- 
munity promise to deliver? The answers differ 
among participants and over time. As Robert White, 
President of the National Academy of Engineering, 
states: "It may be time that we think about whether 
our concern for the support of the science and 
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technology enterprise has diverted us from attention 
to how we can best serve national needs/ 

Congressional Priority Setting 

Since progress begets more opportunities for 
research than can be supported, setting research 
priorities may be imperative for shaping a successful 
Federal research portfolio in the 1990s.^ Tb im- 
prove priority setting at a macro level. Congress 
should hold biennial hearings specifically on the 
state of the research system, including cross-field 
priorities in science and engineering, and the criteria 
used for decisionmaking within the cognizant re- 
search agencies. 

For **objective-oriented'' science and engineer- 
ing that may cr may not cross agencies, such as 
high-temperature superconductivity research, Con- 
gress should allocate resources based on plans to 
attain specific goals. In programs that seek primarily 
to fortify the science base, such as those sustained by 
NSF, Congress could judge progress toward goals 
that reflect the research capacity of the scientific 
community. While objective-oriented programs will 
contribute to these goals, the burden falls largely on 
science base programs to meet the goal of maintain- 
ing the research community. Congressional over- 
sight of the research agencies could include ques- 
tions of how their total research activities and 
specific programs, such as multiyear, capital- 
intensive HK^gaprojects, contribute to expanding 
education and human resources, as well as to 
building regional and institutional capacity. 

If Congress determines that more thorough and 
informed priority setting is required, the executive 
branch must disclose the criteria on which its 
piiorities v/ere set. OSTP is a candidate for this task. 
Building on the Federal Coordinating Council on 
Science, Engineering and Technology (FCCSET) 
mechanical, which presently considers only certain 
cross-agency research topics, OSl^P could also 
initiate broader priority setting. In the executive 
branch. Congress should insist, at a minimum, on 



iterative planning that results in: a) niaking tradeoffs 
among research goals; and b) applying (after scien* 
tific merit and program relevance) other criteria to 
research decisionmaking that reflects planning for 
the future. In addition, since meg^oject costs affect 
the ability of other disciplines to start new, large 
projects, megaprojects are candidates for crosscut- 
ting priority setting,^^ 

Structural improvements to current priority set- 
ting, especially those that facilitate the budget 
process and research planning within and across the 
agencies, would also make the tradeoffs more 
explicit and less ad hoc, and the process more 
transparent. At a minimuiii, agency crosscutting 
budgetary analysis^^ and a separate congressional 
cycle of priority-setting hearings (e.g., biennially) 
could reduce uncertainty and reveal the relationships 
among new and continuing projects, tlie suppoit of 
new investigators by each agency, and the changing 
cost and duration estimates that currently bedevil all 
participants in the Federal research system. 

Congress could also initiate specific changes ^n 
the executive agencies that would increase their 
ability to respond to changing priorities. They would 
include measures that encourage: 1) flexibility, so 
that programs can be more easily initiated, reori- 
ented, or terminated; 2) risk-taking, so that a 
balanced portfolio of mainstream and ' 'long-shot'' 
research can be maintained; 3) strategic planning, so 
that agency initiatives can be implemented as 
long-term goals; 4) coordination, so that crosscut- 
ting priorities can be pursued simultaneously in 
many agencies; and S) experimentation with funding 
allocation mediods, so that new criteria can be 
introduced into project selection and evaluated to 
ascertain the value added to deci.nionmaking. 

It is symbolic that across the Federal research 
system, national policymakers, sponsors, and per- 
formers alike have acknowledged that the funding 
process would benefit from careful consideration of 



'^Roben M. White, "Science, En^ineciin^» and the Sorcerer's Apprentice," PresldeatiAl Adilreu to the NaiiooaJ Academy of Eagiaeoixig, 
WMhiogtOQ. DC, Oct. 2. 1990. p. 12. 

'^Brooks writes. "Ibday nuny of the ume negiUve ligi >i£ tlut existed in 1971 are sgiin evident WLU icieoce recover to experietve • Ekew era of 
prosperity ss il did t>egi]22iinf ia the lite seventies, oi has the day of reclconiog that so many predicted flnilly arrived?' * Harvey Brooks. *'Can Sdeace 
Survive in the Modern Age? A Revisit After Tw^y Yean/* National Forum, vol. 71 , No. 4, fall 1990. p. 33. 

^For example, see Aiissa Rubin* "Science Budget; Hill Must Make Hard Choice Among Big-Mooey Projects/' Congressional Quarterly Weekly 
Report, vol. 49. Feb. 9, 1991. p. 363. 

^*This was a principal recommendation proposed in National Academy of Sciences, op. cit.. footnote 6. 



4 



20 • Federally Funded Research: Decisions for a Decade 



research prioiities, especially at the macro level. 
Whetlier their exhortatio.is lead to clearer research 
agendas (including the suspension or postponement 
of some activities) remains to be seen, and whether 
these investments are balanced, well-managed, and 
yield the desired consequences is hard to judge in 
real time. But surely the policy process is enriched 
by drawing a map of the choices, the benefits, and 
the costs to be incurred by the scientific community 
and the Nation. 

ISSUE 2: Understanding Research 
Expenditures 

Summary 

Many in the scientific community claim that 
the ''costs of doing research'' are rising 
quickly, especially that the costs of equipment 
and facilities outpace increases in Federal 
research funding. The most reliable data are 
available from research agencies, and can be 
analyzed at two levels: 1) total Federal expend- 
itures for research, and 2) individual compo- 
nents of research project budgets, OTA Hnds 
that Federal expenditures for research have 
risen faster than inflation, and more research- 
ers are supported by the Federal Government 
than ever before. Salaries and indirect costs 
account for the largest and fastest growing 
share of these expenditures. However, these 
findings do not truly address the claims ex- 
pressed above, because of the numerous and 
sometimes inconsistent meanings of the costs 
of doing research. 

Most research activities become cheaper to 
complete with time, as long as the scope of the 
problem and the standards of measurement do 
not change. However, advances in technology 
and knowledge are ''enabling": they aUow 
deeper probing of more complex scientific 
problems. Experiments are also carried out in 
an environment driven by coni{)etition. While 
competition is part of the dynamic of a healthy 



research system, conn{)etition drives up de- 
mand for funding, because success in the 
research environment often correlates highly 
with the financial resources of research groups. 

Direct cost containment by the research 
agencies may not be an iq>propriate Federal 
role, although Congress might direct the agen- 
cies to pursue specific measures at their 
discretion and to evaluate their effectiveness. 
Instead, greater cost-accountability could be 
encouraged by the executive branch and Con- 
gress. In particular, the Federal Government 
should seek to eliminate the confusion around 
allowable indirect costs, and develop better 
estimates of future expenditures, especially for 
megaprojects where costs often escalate rap- 
idly. 

Discussion 

Many researchers state as an overriding problem 
that the '*costs of doing research" have risen much 
faster than inflation in the Gross National Product 
(GNP), and Federal expenditures for research have 
not kept pace with these rising costs. Included in the 
costs of research are salaries, benefits, equipment, 
facihties, indirect costs, and other components of 
research budgets. Equipment and facilities are typi- 
cally named as most responsible for increased 
costs.^^ 

However, addi'essing these claims is difficult, 
because it is hard to define what is meant by the costs 
of doing research. Research activities become 
cheaper to complete with time, as long as the scope 
of the problem and the standards of measurement do 
not change. But this is not the way progress is made. 
Advances in technology and knowledge are **en- 
abling": they allow deeper probing of more complex 
problems. This is an intrinsic challenge of research. 

There is an extrinsic challenge as well. Experi- 
ments are carried out in an environment that is 
driven by competition. Competition is part of the 
dynamic of a healthy research system. One sign of a 



^^lii addition lo those cited previously, loe Roben M. Ro*cnzwcig, Prcsidcnl, Aiioclition of Arocrlcwi Univer^itiei. "Address to the President's 
Opening Session, The Gerontologica] Society of AtLetiCA," ^3rd tnnui^l meeting. Boston, MA, Nov. 16. 1990; John H. Dutton tod Lawson Crowt, 
"Setting Priorities Among Scientific Initiitives," American Scientist, vol. 76, No. 6. November- Deccn^ber 1988, pp. 599-603; Albcfl H. Tcich, 
"Scientists tnd fVblic Officials Must Pursue ColUboratioo To Set Research Priorities,** The Hcienttst, vol 4. No. 3, Feb. 5. 1990. pp. 17; and Tina M. 
KoorsbergaiKi Robert L. Park, ''Scientists Muat Face the UapleMant T%ak of Setting Priorities/ ' The Chronicle of Higher Education, vol. 37, No. 23, 
Feb 20. 1991. p. A52. 

*0Sce Janice Long. "Bush's Science Advisor Discusses Declining Value of RAD Dollajs," Chemical a rui E,\gineering News, vol 68, No. 17, Apr. 
2.1. 1990. pp \6-\l, Science The End of the Frontier'^, op. cit , foo'note 14, arid OTA interviews at the University of Michigan aad Stanford University, 
July- August 1990. 
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healthy research system is that it can expand to 
produce more research. "Needs" in the research 
environment are thus open-ended. 

Although competition exists in the research com- 
munity, it does not necessarily drive down costs, as 
would be expected in typical "markets." In an 
earlier era, the chief cost of research was the annual 
salary of the principal investigator (PI). Ibday, the 
PI is often the head of a team with many players and 
access to the latest research technologies. In the face 
of inherent uncertainty about the eventual outcomes 
of research,** sponsors must apply various criteria in 
predicting the likelihood of eventual project success, 
such as access to sophisticated equipment or the 
availability of appropriately trained personnel. 
These criteria are often associated with higher rather 
than lower costs. Success, therefore, often comes to 
those who spend the most (especially if research 
teams are relatively evenly matched). In fact, com- 
petitive proposals are often the most expensive and 
low bids can actually decrease a proposer's chance 
of winning a grant. Because additional personnel 
and sophisticated equipment are seen by sponsors as 
being instrumental in the conduct of research, costs 
are ultimately limited by what sponsors are willing 
to spend. 

Products, or "outputs," of scientific research 
have also traditionally defied measurement.*^ Con- 
sequently, the price of reojarch measured in eco- 
nomic tenns~-the cost per-unit output — is ex- 
tremely difficult to estimate. Analysis using crude 
measures of scientific "productivity" suggests that 
the cost of producing a published paper or perform- 
ing a given scientific measurement has decreased: 
with less than double the investment per year since 
1965, more than double the number of papers are 
published today in academia, and more than double 



the number of Ph.D. scientists are employed in the 
academic sector.*^ By these measures, science has 
grown more productive (and consequently the cost 
per-unit output of research has decreasedV** How- 
ever, there is no metric to compare a unit" of 
today's research with one in the past. 

Thus, • 'Are the costs of research going up? " is not 
a useful question for policy puipjses. Research 
expenditures by the Federal Government -^je 
awarded and accounted for on an annual basis. What 
gets included in these expenditures can be modified 
by adjusting the scale and pace of scientific research. 
Especially for basic research, these factors are 
variable, though the competition for personal and 
institutional recognition pushes Pis toward larger 
teams and more sophisticated instrumentation. In 
mission-oriented science, the rate of research may be 
dictated by pressing concerns (e.g., curbing the 
AIDS epidemic is desired as quickly as possible). 

For policy puiposes, research costs equal expen- 
ditures: if the Federal Government provides more 
funds, "costs" will go up accordingly. A more 
useful policy question might be: "Is Federal spend- 
ing on individual components of research project 
budgets reasonable?" The Federal Government vill 
tend to have a different point of view on this question 
from the research performer. OTA has explored both 
perspectives. 

Incomplete and murky data on research expendi- 
tures complicate questions on the costs of research. 
Analysis of Federal expenditures for the conduct of 
research must factor what Federal agencies are 
willing to spend for personnel, facilities, and instru- 
mentation, while analysis of expenditures by re- 
search performers is confounded by the expenditure 
accounting schemes that vary from research institu- 



'Sec exiinple, Wchud NeUon^ "nie AUocdon of Research tod Developmeni Resourcei; Some Pivblans of PubUc Policy." Economics of 
RejearchandDevelopment, Rlclurd 1>bout (ed.) (Columbui. OH: Ohio Sute Univcriity Pmt. 1965). pp. 2S8-308. NeUon poinu o«t L " res^^ 
Md dcvelopmcitf huecooomlc v^ue bectuie thi infonnatiofl penniu people to do thiagi better, and lometimei to do thiagi that they did not know 

sr29S?25X"L,s'iadnp ct"^^ • • • • 

•*^PuWlihodp«p«M and patei«« have been ttiedMprojilc«.bi'J they cannot be itai^ Cozieni."Liierature-Ba«!dD«iiMin»is««rch 
Evaluadoo: A Maaag«'i Guide to BibUometrlci." final npoft to ttie National Science PoundaiV&StriMwV^^^^^^^^^^^^ 

il^fJ^ If^JJ-SJKc'^i'^ "If^ ■;Bibliometrtc..'Mflflw/i«*Wnv ofMomatkon Scitnct and Technology, vol. 24. 1989. pp. 
119-186; and oo the latter, National Science Board, op. cit., footnote 12. tablei 5-17 and 5-30. 

.u^rS^^L^If. ^ ddinitjooj of reie^ch output, the productivity of reiearch relative to other economic activitiei might »tiU be 

•taipant. Ecoaomiil WUliam Baumol explain* that reaeareh, due to the price of labor rather than increase* in lu pitKfuctivity ha* an faAmn, 
tendency to rl*e la cort and price. penUleotly cumulatively, reUtive toO* co.t. and price, of oiSStT " ifctSu that' 

coDKjp^ace nuy be an Impediment to Mlequate funding of RAD activity, that U. to a tevel of ftmdlag c<S^rtIiiT rcqu^^ Ttc^^ 
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Figure S—Estimated Cost Components of U.S. Academic R&D Budgets: 1958^ (In billions of 1988 dollars) 
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1080). flgura2-43. 

NOTE: Conitant doVara wara calcutatad utlng tha ONP ImplWt Prtoa 
Daf later. 

DEFtNmON Of TERMS: Eatlmatad paraonnal ooati for 9$nlor wohntktB 
and gmduMts aiiMMnta Induda salarlaa and fHnga banafita. auch at 
Inturanoa and ratlramant oonlributlona. Of/w d/raof ooata Induda 
auch budgat Itama aa nfwtariala and auppllaa. traval, auboontractora, 
oomputar aarvloaa. publloatlona. oonaultanta. and partldpant aup^ 
ooata. Indimot ooata Induda ganaral admlnlitratlon, dapartmant 
admlnlatrallon.bulUIngoparatlonandmalntaoanoa.dapradatlonand 
usa, iportaorad-raaaarah projada admlnlatratlon, llbrarlaa, and afu- 
dant-aarvicaa admlnlatratlon. Equlpmmf OMta Induda: 1) raportad 
axpandlturaa of aapamtdy budgatad currant fundi for tha purahaaa of 
raaaarehaqulpmant,and 2)aatlmatad capital axpandlturaa forflxador 
bullt-ln raaaarch aqMipmant. F9oilltl9$ costa Induda aatlmatad capital 
axpandlturaa for raaaarch fadlltWa, Indudtng fadlltiai oonatoidad to 
houaa adantlflc apparatus. 
DATA: National Sdanoa Foundation. Division of Poltcy Raaaarch and 
Analysis. Databasa: CASPAR. Soma of tha data within thia 
databaaa ara asllnuitaa, Inoorporalad whara lhara ara dtacontlnul- 
tlaa wtthln data aartaa or gapa in data coHactlon. Primary data 
sourca: National Sdanoa Foundation. Division of Sdanoa R^aouroa 
Studiaa, "Survay of Sdantlflc and EnginaarVig Expandtturaa at 
Univarsltlaa and Cdlagaa"; Natk3n«] tnatltulas of Haatth; Amarlcan 
Assodatlon of Unl varsity Profaasors; National Aasodatk3n of Stata 
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tion to research institution.*^ In addition, much of 
the cunent debate over rising expenditures takes 
place within a context of agency budget constraints 
and pressures felt by research performers. 

The most reliable data on Federal research expen- 
ditures are available from re: h agencies, and can 
be analyzed at two levels: 1; ^ Federal expendi- 
tures for research, and 2) individual components of 
research project budgets. OTA finds that total 
expenditures on individual components of grants 
have risen over inflation, but not nearly at the rate for 
total Federal expenditures for research (see figure 8). 
Instead, growth in the size of the research work force 
supported by the Federal Government seems to 
account for the largest increase in Federal research 
expenditures. Also, the largest component increases 
of research project budgets are for salaries and 
indirect costs. 



Trends in Components of Total 
Federal Research Expenditur 

Analyzing Federal expendiuuej for specific line 
items of research budgets reveals interesting trends 
(again see figure 8), First, reimbursements for 
indirect costs are the fastest growing portion of 
Federal research expenditures. Indirect costs is a 
term that stands for expenses that research institu- 
tions can claim from the Federal Government for 
costs that cannot be directly attributed to a single 
research project, i.e., they are distributed over many 
investigators who share research infrastructure and 
administrative support. Federal support for indirect 
costs has increased since the 1960s, with the largest 
increases in the late 1960s and the 1980s. In 19S8, 
indirect cost billings comprised 10 to 15 percent of 
Federal academic K&D fundings By 1988, that share 
had risen to roughly 25 percent.^ In addition, some 
agencies allow more than other agencies in indirect 
costs. For example, in 1988, the indirect cost as a 
per: ent of the total R&D expenditures allowed at 



^For an aiiempt co compare expeodlcuies U two public and two private univerticiea auocUted with the perfonnaoce of Natiotul Sdcoce 
FoundaUoo-funded rcicarch, »cc G.W. Daughman, * Impaa of Inflaltoo oo Rescartli Bxpeaditurei of Selected Academic Disciplinca 1967-1983/ ' report 
to Ok: Ntitooai Science Foundation and the National Center for Educational SiaU$tic», NSF/H-N 80178 15, Nov. 8. 1985. Alio tee Daniel E. Koihland, 
"TbeUndctiideofOvertiead«''5de/k:e. vol.249, May 11, 1990. p. 3; and "The Overliead Question,*' letters in response to Koshland*! editorial, £de/u:e, 
vol. 249. July 6. 1990, pp. 10-13. 

^National Science Foundaiion, The State of Academic Science and Engineering (Washington, DC: 1 990), p. 1 2 1 . 
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PMo cndh: Bob Kdmbteh, UnlvtnHy of Michigan 

TheM sdemists are In an Ion beam laboratory at the 
University of Michigan. Research often requires state-of- 
the-art eqt^pment. 

NIH was 30 percent, whereas it was less than 24 
percent for NSF (a proportion unchanged since the 
mid-1980s) *7 

Second, increasing numbers of investigators and 
rising salaries (and the benefits that go with them) 
have driven up the price of the personnel component 
of direct costs. University personnel speak of the 
increased competition for faculty with other sectors 
of the economy, and note that faculty salaries have 
been rising significantly over inflation during the 
last decade. The average total compensation (sala- 
ries and benefits) for academic Ph.D.s in the natural 
sciences and engineering increased from $59,000 



(1988 dollars) in 1981 to more than $70,000in 1988. 
In the same period, the number of full-time equiva- 
lent scientists and engineers employed in academic 
settings rose steadily from about 275,000 to ahnost 
340,000.'« 

Third, Federal support for academic research 
equipment alone increased from $0.5 billion in 1968 
(1988 dollars) to $0.9 billion in 1988. Despite 
pronounced increases and improvements in equip- 
ment stocks in the 1980s, 36 percent of department 
heads still describe their equipment as inadequate (to 
conduct state-of-the-art research). This is in part due 
to the reduction in the obsolescence time of equip- 
ment and instrumentation use since the late 1970s.^' 

Finally, the Federal share of all capital expendi- 
tures for academic facilities (which include both 
research and teaching facilities) has never topped 
one-third. Now it is less than 10 ptercent.^^ For 
university research facilities alone, the Federal 
Government provided an estimated 11 and 16 
percent, respectively, of private and public univer- 
sity capital expenditures in 1988-89. The govern- 
ment also supports research facilities through depre- 
ciation, operation, and maintenance charges 
accounted for in the indirect cost rate. In 1988, the 
Federal Government supplied nearly $1 billion to 
s ipport university infrastructure. Ahnost 20 percent 
WLS for facilities depreciation, while the rest was 
recovered for operation and maintenance costs.'^ 

Academic administrators claim that with growing 
frequency, aging laboratories and classroom build- 
ings falter and break down ,'2 and many claim that 
facility reinvestment has not kept pace with growing 
needs. However, the picture is not clear. For 
example, when asked by NSF, a majority of the 
research administrators and deans at the top 50 



^'Ibid.. pi 42: and AisocUUon of American Univcnitici. Indirect Costs Associated With Federal Support of Research on Un .versity Campuses; Some 
Suggestions for Change (Washington, DC: December 1988). 

«Oovernraa|.Unlvcniity-Induttry Research RoundUble. Science and Technology In the Academic Enterprise: Statuz, Trends, and Issues 
(Washington. DC: National Academy PreJi. October 1989). pp. 2-34 and 2-47. based on National Science FoundaUon dau. 

<»NatioQal Science FoundaUon. Academic Research Equipment in Selected Sciencel Engineering Fields: 1982-83 to 1985-86, SRS 88-Dl 
vWAsiuii£i(ra, DC: June 198o). 

.m.*!^ ^.'f ''^i'^ii"'^'^""*' f«lli«l«»fun<l*comc&x.m IheSlatc. and 30 percent from bond iuues. For private univmiUc*. roughly 
?r 'T '^^'^'^ Govwnmfflt. whUe another one-third ii from donations. S« Michel Davcy. Bricks Mortar: A Swn^and 
Analysis of Proposals to Meet Research Facilities Needs on College Campuses (Washington. DC: Coogreaiiooid Research Service. ynrT^ 

In . •iS^'Jf ^ l^*' "."^ ^'^^ "PP"" °^ university infrutructurc grew ova 70 percent in itaJ texma. TTieie figlirea are nrescnted 

fa Enhancing Reiearch and Expanding the Human Frontier." op. cit.. footnote 26. pp. 61-62. document further state. 

not Lad an apparent effect on the abUity of univer«ities to accept Federal rwarch fund*.* * o7» ai«uc« . . . na« 

unuKAiy, / He (.nromue of Higher Education, vol. 37. No. 6, Oct. 10. 1990. pp. Al, A34. 
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research universities replied that their facilities were 
•*good to excellent/' whereas a majority of the 
research administrators and deans in the schools 
below the top SO estimated that their facilities were 
•^fairtopoor/'" 

The crux of the facilities problem is that research 
and academic centers can always ure new or 
renovated buildings, but how much is enough? Even 
though **need*' may not be quantified in the 
different sectors of ilie research enterprise, a demand 
certainly exists. For example, when NSF solicited 
proposals for a $20 million program in 1989 to 
address facilities needs, it received over 400 propos- 
als totaling $300 million in requests.^ 

Federal Policy Responses to Increased Demand 

Many Federal agencies have experimented with 
grant-reducing measures, such as the salary caps 
required by Congress and temporarily imposed by 
NSF and NIH, the ceilings on indirect costs currently 
in place at USDA, the elimination of cost-blind 
reviews of proposals in some research programs at 
NIH, the limitation of funds supplied in new grants 
to researchers with multiple Federal grants at the 
National Institute of General Medical Sciences, and 
the institution of fixed-price grants in some NSF 
programs.^^ Congress could pursue permanent 
grant-reducing measures to slow or limit inaeases in 
research expenditures on individual research grants. 
However, it may not be an appropriate Federal role 
to dictate specific allowable costs in research 
projects. In general, allowing market forces to 
determine costs has been a tradition in Federal 
policy. 

Instead, greater cost-accountability could be en- 
couraged. One benefit of cost-accountability could 
be incentives for performe) j spend less than what 
was targeted in project but ' ets, and greater flexibil- 



ity in expenditures for performers (e.g., researchers 
could be encouraged to use the money saved one 
year in the next year, a so-called no-cost extension). 
Within such cost-accountability measures. Congress 
might also direct the agencies to experiment with 
cost-containment schemes and to evaluate their 
effectiveness. 

Greater cost-acountability is especially important 
in the calculation of indirect cost rates. At present, 
the guidelines for calculating costs are detailed in 
conjunction with 0MB Circular A-21 and have been 
in force since 1979. Every major research university 
has an indirect rate established for the current fiscal 
year for recovery of costs associated with sponsored 
research. These rates have evolved over many years 
as a result of direct interaction and negotiation with 
the cognizant Federal agency. There is a wide range 
of indirect costs rates among universities, with most 
noticeable differences between public and private 
institutions (rates tend to be higher at private 
institutions). Rates vary because of: 1) significant 
differences in facilities-related expenditures, 2) un- 
derrecovery by some universities, 3) imposition of 
limits by some government agencies in the negotia- 
tion process, and 4) diversity in assigning compo- 
nent expenditures as direct or indirect.^ 

However, confusion around what is contained in 
the indirect cost rate is getting worse, not better. This 
reflects, in part, the difficulty of separating expendi- 
tures along lines of research, instruction, and other 
functions .^'^ Recent investigations by the Office of 
Naval Research and the House Committee on 
Energy and Commerce have also uncovered signifi- 
cant variation in the accounting of indirect costs by 
the cognizant Federal ::5cncies and i^search univer- 
sities.^^ Hiese differences should be sorted out, and 
more explicit and understandable guidelines de- 
vised. 



"Nilional Scieocc FoundtUon. Scientific and Enginetring Research Facilities at Universities and Colleges: 1 988, NSF 88-320 (Wtthington. DC; 
September 1988). p. 26, 

^Stt Jeffrey Mervii. '•Insiilulioai Respond in Lvge Number* to Tiny Ftciaiici Prognun at NUt NSF,* ' The Scientist, vol. 4, No. 8. Apr. 16, 1990, 
p. 2. 

^Por i discuuioQ of various options, see Barbam J. Cuiliton. "NIH Readies Plan for Coit Containment, '* Science, vol. 230, Nov. 30. 1990. pp. 
1 198-1 199; and Colleen Cordcs. "Universities Pcat That U.S. Will Limit Paymenu for OvertKid Costa Incuned by Resevcbers/* The ChronicU of 
Higher Education, vol. 37, No. 3. Nov. 21. 1990, pp. A19. A2L 

•^Aisocladon of American Universities, op. cit.. footnote 47. 

^^Elcaoor C. Tbomas and Leonard L. Lederman« National Science Foundation, Directorate for Scientific. TDcbnologicai. and International Affairs, 
''Indirect Cosu of Federally Funded Academic Research,** unpublished paper. Aug. 3, 1984. p. 1. 

^Sce MarcU Barinaga, "Stanford SaiU Into a Storm,'* Science, vol. 250. Dec. 21, 1990, p. 1651; '•Government Inquiry,** Stanford Observer, 
November-December 1990, pp. 1, 13; Colleen Cordes, "Conceding 'Shortcomings.* Stanford Tb Forgo $500,000 in Overhead on U.S. Contracts,*' The 
Chronicle of Higher Education, Jan. 30, 1991. vol. 37, No. 20, pp. A19, A22; and Colleen Cordes. ^'Stanford U. Embroiled in Angiy Controveny on 
Overhead Charges/* The Chronical of Higher Education, Feb. 6. 1991. vol. 37, No. 21, pp. Al. A20-A21. 
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It is also important to stress accuracy in develop* 
ing estimates of costs for megaprojects. When the 
Federal Government •'buys" a megaprojcct, the 
initial investment seems to represent a point of no 
return. Once the go, no-go decision has been made 
at the national level, the commitment is expected to 
be honored. However, criteria for consideration in 
the funding of a science megaproject could conceiv- 
ably include: startup and maintenance costs, cost of 
unanticipated delay, cost of users' experiments, and 
likely changes in the overall cost of the project from 
initial estimate to completion. Some estimates for 
science megaprojects double before the construction 
is even begun, and costs of operating a big science 
facility once it is completed are sometimes not 
considered. 

Megaprojects will always be selected through a 
poUtical process because of their scale, lumpiness, 
and incommensurability. Since their costs, espe- 
cially in following years, affect other disciplines* 
abilities to start new, large projects, megaprojects 
could well be considered as candidates for crosscut- 
ting, priority-setting analysis before the practical 
point of no return. As the National Academy of 
Sciences* report on budget priorities reminds: 
''...it is necessary to specify the institutions, 
individuals, and orsanizations that will be served; 
[and] the costs ... of the program.*'^ The cost of 
investment for the Federal Government is an impor- 
tant criterion to apply to all scientific research, 
including megaprojects. 

Performer Expectations 

Not all problems in research costs can be ad- 
dressed by the Federal Government. Many research- 
ers point to higher expectations, which require more 



spending, and competition in the university environ- 
ment. In the academic environment, researchers are 
asked today to publish more papers, shepherd more 
graduate students, and bring in more Federal funding 
than theii predecessors.^^ If they do not meet these 
expectations, some report a sense of failure.® This 
is true even if they have succeeded, but liot by as 
much or as quickly as they had hoped. 

lb boost research productivity and to compete 
with other research teams, facul^ attempt to lever- 
age their time with the help of postdoctoral fellows, 
nontenure track researchers, and graduate students 
who are paid lesser salaries. Due to the shortage of 
faculty positions for the numbers of graduate stu- 
dents produced, young Ph.D.s have been willing to 
take these positions in order to remain active 
researchers. This availability of ''cheap labor'' is 
seen by many senior researchers and their institu- 
tions as the only way they can make ends meet in 
competing for grants.^^ This is a trend toward an 
''industrial model,'* where project teams are larger 
and responsibilities are more distinct within the 
group.^ While the expenditures charged to an 
individual grant may be less (since more grants may 
be required to support the diverse work of the group), 
the overall cost of supporting a PI and the larger 
group are greater.^ 

Some experiments have been attempted on U.S. 
campuses to temper the drive for more research 
publications (as a measure of productivity). For 
example, at Harvard Medical School, faculty are 
allowed to list only five publications for considera- 
tion in tenure reviews, with similar numbers set for 



^or example, sec Kunlz, op. cit.. footnote 31; and Da^id P. Hamilton, ''The SSC lUces on a Life of lu Own,'' Science, voJ. 249. Aug. 17, 1990, 
pp, 371-372, 

National Acadesmy of Sciences, op. cit., footnote 6, p. 1 1. 

^'This if eitpecially true in entrepreneurial research areas such as biotechnology. See Henry Etzkowiu, ' *Bntreprcneurial Scienlists and CatrepreneuriAl 
Universities in American Academic Science.*' Minerva, vol. 21, summer-autumn 1983, pp. 198-233. 

^Science: The End of the Frontier? op. cit., footnote 14. 

^Labor ecoiK)mist Alan Feebler, Executive Director, Office of Scientinc and Engineering Personnel, National Research Council, writes: 
"... personnel cosu constitute roughly 45 percent of total costs and ... this percentage bu remained reasonably stable ov<:r time. Given that salaries 
of faculty (i.e.. principal investigators) have been rising during the 1980a. this suggeaU that the staffing pattern of research projecu has been changing, 
with the input of Pis decreasing relative to . . . other, less expensive resources. There is some evidence to support this hypothesis in the report of OUIRR 
[Govemmem-Unlversity-Industiy Research Roundtabk] ...[that] finds in academia an increasing ratio of noofaculty to faculty." personal 
communication. Nov. 15, 1990. See Govemment-Univcrsity-Industry Research Roundtable, op. cit, footnote 48. 

^Elsewhere this hu been called the ' 'industrialization* ' of science, or "... a new coUtctivixed form in which characteristics of both the academic 
and industrialized modes are intermingled." See John Ziman, An Introduction to Science Smdies (Cambridge, England: Cambridge University Press. 
1984). p. 132 (elaborated below). 

^Noted at OlA Workshop on the Costs of Research and Federal Decisionmaking, July 19. 1990. 
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other promotions.^ Thus, the quality and impor- 
tance of the candidate's selected set of papers is 
stiessed. though measuring these characteristics 
remains controversial.^^ However, strong incentives 
militate against reducing research volume. Most 
overhead is brought into the university by a small 
number of restcU*ch professors. (At Stanford, 5 
jxjrce of I* iHculty bring in over one-half of the 
indirect i j^ars.) Any measure that would 
reduce gram - v/ards and publications produced by 
tliese investigators would deprive tlie university of 
revenues. In fact, many universities in tight financial 
straits try to maximize tlie level of research volume. 

Tlie Federal Government nust seek to understand 
better the trends in eXj^enditures in the rese<u'ch 
environment — especially variations across institu- 
tional settings — and craft government policies to 
allocate resources effectively. Reliable analyses of 
reseiu-ch expenditures at all of the Federal agencies 
are not available. Future studies of expenditures 
should look not only at the economic forces that 
increase (and decrease) research expenditures, but 
also at the sociology of research organizations, 
including the demography of research teiuns and 
institutional policies for sponsored projects.^''^ 

Federal agencies clearly must understand increas- 
ing demands to fund research, as research universi- 
ties and laboratories are an invaluable resource for 
the United States. Devising mechanisms for coping 
with research expenditiu*es is one of the central 
challenges to the Federal system for funding re- 
search in the 1990s. 

ISSUE 3: Adapting Education and 

Human Resources To Meet 
Changing Needs 

Sufnrmry 

Three issues are central to education and 
human resources for the research work force: 

1. Recent projections of shortages of Ph.D. 
researchers in the mid- 1990s have spurred 



urgent calls to augment Ph.D. production in the 
United States. OTA believes that the likeli- 
hood of these projections being realized is 
overstated, and that these projections alone are 
poor grounds on which to base public policy. 
For instance, they assume continued growth in 
demand in both academic and industrial sec- 
tors, independent of the level of Federal 
funding. In both this and previous OTA workt 
however, OTA has indicated the value to the 
Nation — ^regardless of employment opportuni- 
ties in the research sector — of expanding the 
number and diversity of students in the educa- 
tional pipeline (K-12 and undergraduate) for 
science and engineering, preparing graduate 
students for career paths in or outside of 
research, and, if necessary, providing retrain- 
ing grants foi researchers to move more easily 
between research fields. 

2. Total participation in science and engi- 
neering can be inaeased if the opportunities 
and motivation of presently underparticipating 
groups (e.g., women, minorities, and research- 
ers in some geographic locations) are ad- 
dressed. Federal legislation has historically 
played an important role in recruiting and 
retaining these groups. Also, ••set-aside" pro- 
grams (which offer competitive research grants 
to targeted groups) and mainstream discipli- 
nary programs are tools that can enlarge, 
:iustain, and manage the diversity of people and 
institutions in the research system. 

3. Research in many fields of science and 
engineermg is moving toward a larger, more 

industrial*' model, with specialized responsi- 
bilities and the sharing of infrastructure. In 
response, the Federal Government may wish to 
acknowledge changes in tlie composition of 
research groups and to enhance the opportuni- 
ties and rewards for postdoctorates, nontenure 
track researchers, and others. 



'^The Nalioniil Science FfundaUoiiuJso now liniilJi ihc niunlxrrof pubhcaljons ii w\li consider, as evidence of an applicant's track record, in reviewing 
gr;inl proposals. Sec David F Hamilton. "Pubiislmig By -and For?- (he NunU>crs." Saenct. ''ol 250, Dec 7. 1990. pp. r^3l-1332. 

'•''SccN.L Cellcretiii . "Ltfcumc Citation Rales to Compare Scicnlisls' Woik/' Sth ml Scuncf Rf search, vol 7. No 4. i978.pp 345-36.*); and A. L. 
Porter el al , "CilalU)a% anU Scientific Progress. Comp;mng Bibliomeinc Measures Willi Scicnbsl Judgments." Scientanietncs vol 13, 1988, pp 
103124. 

''^OTA iniervicws ai Stanford University. Aug 21. WO 

^SccSusanP. Co/^ciwctal (ods.) The Research S\ \tt'ffu/i IranMihtn, i'tni.crKUu^^ ni a NAI C) Advanced .Stud> lastiiutc. 11 Cuxco. Italy. ( ki 
1*M9 (Uordrtxht. Holland Kluwer. 19X)) 
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Discussion 

The graduate science and engineering (s/e) educa- 
tion system in the United States, especially at tne 
doctoral level, is the envy of the world. Foreign 
nationals continue to seek graduate degrees from 
U.S. institutions at an ever-growing rate.''° From 
1977 to 1988, the number of Ph.D.s awarded in s/e 
by U.S. universities increased by nearly 50 percent'' 
(for a breakdown by field and decade, see figure 9). 
This exemplary production of Ph.D.s continues a 
noble tradition abetted by Federal research and 
education legislation. 

With passage of the National Defense Education 
Act of 1958 (Public Law 85-864) in the wake of the 
Sputnik launch, the Federal Government became a 
pivotal supporter of pre- and postdoctoral science, 
engineering, and indeed, non-s/e students. '^2 f^^^, 
tional programs were soon established by NSF, 
NASA, NIH, and other Federal agencies. This period 
of growth in Federal programs offering /<?//ow5/h/).t 
(portable giants awarded directly to students for 
graduate study) and traineeships (grants awarded to 
institutions to build training capacity) was followed 
by decreases in the 1970s.''3 decline was 

offset by the rise in the number of research 
assistantships (RAs) for students awarded on Fed- 
eral research grants to their mentors. 

During tlie 1980s, RAs became the principal 
mechanism of graduate s/e student support, increas- 
ing at f/ percent per annum since 1980, except in 
agricultural sciences where RAs have actually de- 
clined. (A comparison of the types of graduatr 
student Federal support, 1969 and 1988, is prewnted 



in figure 10.)'* This trend is consistent with the 
growing '•research intensiveness" of the Nation's 
universities: more faculty report research as their 
primary or secondary work activity, an estimated 
total in 1987 of 155,000 in academic settings.'^ 

Thus, the Federal Governmrnt has historically 
played both a direct and indirect role in the 
production and employment of s/e Ph.D.s. Both as 
the primary supporter of graduate student stipends 
and tuition, and as a patron, mainly through research 
grants, the Federal Government has effectively 
intervened in the doctorate labor market and helped 
shape the research work force. 

Supplying the Research Work Force 

The U.S. graduate research and education system 
trains new researchers and skilled personnel for all 
sectors of the Nation's work force (and arguably for 
some countries abroad). Since 1980, NSF estimates 
that the total s/e work force (all degrees) has grown 
at 7.8 percent per year, which is four times the annual 
rate of growth in total employment. Scientists and 
engineers represented 2.4 percent of the U.S. work 
force in 1976 and 4.1 percent in 1988.'<* 

While new s/e Ph.D.s have traditionally been 
prepared for faculty positions in acadcmia—almost 
80 percent were employed in this sector in 1987''— 
in broad fields such as engineering and disciplini s 
such as computer science the demand for technicrj 
labor outside of academia is great. Other fields, liJte 
chemistry, benefit from having a large set of 
potential academic and industrial employment op- 
portunities. This diversity makes any labor market 



''N«Uon«J Science Foundilion. ic * and Engineering Do'joraies: 1960-89. NSF 90-320 (Waihington. DC- 1 990) table 1 
^^AuocUlion of AmericM Univcnitiei. The PhD. Shormge: The Federal Role Cf^uhingr. o. DC- 11 1 990) pp 1 «,16 

OTA-SET.377 (Wa^lngton, DC. U? (^van^LS^X^J^^S^^ "nd Engineers: Grade School to Grad School, 

in Science and Engine^ng. NSP 90-301 (W^^.^TS l«0)f 23? J '^""'^ Foundlloa, Won,^ and Minorities 

..^"uSst^^riS^S;'" • " I---' -lenusu «d c.plov«, 

'«IWd.. p. 67. Among Ph.D.s. tlic rtUo ol omployed iciemis(s to englucws is 5 to 1. 
'^Ibid.. app. uble5-19. 
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Figure Percontaga Distribution of Doctoratts by 
Sclenca and Englnaaring Flald: 1960^9 (by dacada) 
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statistical''.:^, tabla 1. 



f.uid aid its forecasiing difficult, but the majc. 
components can be analyzedJ^ 

Based on changing demographics and historical 
trends in baccalaureate degrees, some studies lave 
projected that the scientific community will face 
severe sho iage in its Ph.D. research work force 
during the 1990s.''^ However, there are pitfalls in the 
methodologies employed in the^ projerrions of 
Ph.D. employment demand.^ Predicting the de- 
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Founds tlon. Urmkmf Stucknt Support and MMnpowr 
Bourcmiln QrsduMfB Sd^nc^ BckKathn, Fail 1969, N8F 70^0 
(Waahb^n DC: 1970). tabla C-lla. 



mand for academic researchers must also account for 
enrollment and immigration trends, anticipated ca- 
reer shifts and retirements, and the intentions of new 
entxai as well as shifting Federal priorities and 
available research funding. All of these are subject 
to change, and may vary by institution, field, and 
region of the country .^^ In addition, OTA questions 



TtPor exsmples, •« Elk«a L. CoUlns, *'Moc^<Jog the Sc" nUfU: and Technical SufttJ^s Rtqui-inaitt of die Aoialcaa Ecooooiy / * Science and Public 
Pol ^. vol. 15. No, 5, October 1988, pp. 333-342; tod N «uoaal Research CouncU, The Effects on Qualify rf Adjustments in EnginetHng iMbor Morhets 
(Watbiiigtoa. DC: Naliom ' Academy Pttts. 1988). 

'''See Rktjfd C ACkia m, * 'Supply sod Demand for ScicnUsu and Engineers: A Ns' .ooal Crisis in the Making/* Science, vol. 246, Apr. 27, 1990, 

425-432. 

^^'ludeed. shortages may not be ihe biggest coocem. Changes in dctuographic composiiifXi and quality of graduaies miy be more problemttic. For 
a dlKussion* see Howsrd P. llKkman, "Supply. Human CspiutL and the Average QuaLty Level of the Seience and Engineering Labor Force," 
Economics of Education Review, vol. 7, No. 4, 1988, pp. 405-421. 

•'For example, ice Ted LK. Youn, " Studies of Acad-mic Maikeu and Careen:: An Historical Review,' ' Academic Utor Markets and Careers, D.W. 
Breneman and Ted LK. Youn (eds.) (Pbihdeiphia, PA: The Palmer Press. 1988), p^' 
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the ability of statistical analyses to predict future 
demand for s/e Ph.D.s, especially as responses to 
market signals and other societal influences are 
known to adjust both interest and opportunities. 
Even without the prospect of a slackening economy 
in the 1990s, such projections would be unreliable. 
Given the track record of these forecasting tools, 
they are poor grounds alone on which to base public 
policy.'^ 

Noting the uncertainty of projections, OTA fmds 
that concentration on the preparedness of the pipe- 
line to produce Ph.D.s (i.e., increasing the number of 
undergraduates earning baccalaureates in s/e) by 
introducing flexibility into the system is the most 
robust policy. If shortages begin to occur in a 
particular field, not only should graduate students be 
encouraged to complete their degrees (i.e., reducing 
attrition), but prepared undergraduates should be 
induced, through various proven Federal support 
mechanisms, to pursue a Ph.D.'' Those scientists 
who would have otherwise left the field might stay 
longer, thos<) who had ah-eady left might return, and 
graduate students in nearby fields could migrate to 
the field exj^eriencing a shortage. If shortages do not 
materialize, then the Nation's work force would be 
enhanced by the availability of additional highly 
skilled workers. 

OTA believes there are initiatives that maintain 
the readiness of the educational pipeline to respond 



to changing demands for researchers and that 
enhance the diversity of career opportunities — 
sectors and roles — for graduates with s/e Ph.D.s.^ 
Congress could urge NSP and the other research 
agencies to intensify their efforts to maintain a 
robust educational pipeline for scientific researchers 
(and to let the labor market adjust Ph.D. en^loy- 
ment). Funding could be provid^ for undo'graduate 
recruitment and retention programs, for grants to 
induce dedicated faculty to teach undergraduates, 
and for the provision of faculty retraining grants.^' 

Expanding Diversity and Research Capacity 

Trends in the award of s/e degrees attest to 20 
years of steady growth in human resources (sec 
figure 11). These data are a sustained record of 
scientific education at the Fli.D. level. However, the 
benefits of this education do not accrue equally to all 
groups, and therefore to the Nation. Women and 
U.S. racial and ethnic minorities, despite gains in 
Ph.D. awards through the 1970s and 1980s, lag the 
participation of white males. Relative to their 
numbers in both the general and the undergraduate 
populations, women and minorities (and the physi- 
cally disabled) are undeiparticipating in the research 
work forcc.*^ Meanwhile, foreign nationals on 
temporary visas are a growing proportion of s/e 
Ph.D. recipients (and about one-hidf are estimated to 
rei -ain in the United States).^^ 



*20TAre«cbed this conclujioo after eximlning the perfonnaace of vnrioui Dxxleli of acjuiemlc .\a'i induitiiil labor markeu . Se« Office of Technology 
AMe»iroem, op. cil., footaote 72, especially chs. 3 and 4. ReceM indepeadeni conflnnaJion of (his cowluilon appear* la Alan Fech«er, "EiwiiieeriM 
Shortages and SbortlaUi: Mythi and Realitie*," The Bridge, fall 1990, vol. 20. pp. 16-20. 

"See Office of Technology Asieionenl. op. clL. footnote 74. 

"See two repotti: U.S. Congreii. Office of Teehoology Ajiessment. Elementary and Secondary Education for Science and Engineering, 
OTA-TM.SET-41 (Washington, DC: U.S. Oovemmeat Printing Office, December 1988): and Higher Education for Science and Engineering, 
OTA-TM-SET-52 (Waihlngtoo. DC: U.S. Oovcromeot Printing Ofike. March 1989). . 

»»The Natiooal Science Foundation, ai preicrlbed in their eubling legiiUtion, u cquaUy responsible for science education and the support of the 
Nation's basic research. It has grKlually expanded its prognnu. long focused on the graduate end of the pipeltoe. to address issues In undergraduate 
and K-12 education. For example, see National Science Foundation, Research on Key fsjues in Science and Engineering Educarion: Targeted Program 
Solicitation, NSF 90-149 (Washington. DC: 1990). Perhaps faculty retraining programs, both to highlight changes in educatlooal strategies and 
developmenu in research, should be considered. Retraining hu been acknowledged u In^portant for maintaining the engineering work force and 
retraining grants have been provided In some prognnu within the Depanmem of Defease and other agcnclet. Additional research retraining gnnU could 
certainly be financed by the research agencies and pertu^a adminUiered through the Federal laboiato^^ 

to universities that wish to Improve the classroom (i.e., undergraduate) teaching of its faculty. See Naitonal Research CouncU, op. clt.. footnote 78; and 
. !^ !f ' ' Challenges and OppoctuniUes, ' ' Human Resources in Science. andTechnology. Improving V S. Competitiveness, Proceedings 

of a Policy Symposium for Oovemment, Academia. and Industry, Mar. 15-16, 1990, Washington. DC. Betty Vetta and XUeanor Babco(eds) 
(Washington, DC: Coaunlsslon on Professioaal In Sclrxe and Tecbaology, July 1990), pp. 92-99, 

•♦Degrees alone tell an incomplete story of future si^ly of sciemlsu and engineen . For example, college attendaiKe rates of 18 - to 2 1 -year-olds vary 
by gender and race. Since 1972, 33 to 40 pt^cent of whites of both sexes in cohort have attended coUege with BUck rates In the 23 to 30 percent 
range. By 19S8. female aneodance exceeded that of males and was rising, whereas male attauUnce of both races peaked In 1986-87 and declined 
thereafter. See Natloaal Science Board, op. dt., footnote 12. p. 30. figure 2-2. u«,imw 
, "J* " o***^*"^' *^ Commission oo ProfessknaU In Sdeoce and Technology. Measuring National Needs for Scientists to the Year 2000 Repon 
of a Workup. Nov. SO-Dec. 1. 1988 (Washington, DC: July 198s\ pp. 20-24. For more oo graduate engineering education, see EUnor Bart>ir etT 
Choosing Futures: U S. and Foreign Student Views of Graduate Engineering Education (New York, NY: Institute of Intemadoaal Education. 1990)! 



ERIC 



30 • Federally Funded Research:. Decisions for a Decade 




Photo ctodit: Bob KaJmbttoh, lMm$tty of Mkhlgan 

These students are In a laboratory at the University of 
Michigan. Laboratory dasses are a crudal part of 
undergraduate education In the physl^^i sciences. 



Increasing the participation at all educational 
levels in s/e by traditionally underrepresented 
groups is a challenge to the human resources goals 
of the Federal research system. Enhancing the 
participation of targeted groups at the Ph.D. level 
will be particularly difficult, as the research work 
force adjusts to changing fiscal conditions and 
funding of research. As OTA found in an earlier 
study, . . equal opportunity for participation in 
higher education and in research for all groups is a 
long-term social goal that will be achieved only with 
steady national commitment and investments."^^ 
Congress could amend the Higher Education Act 
(reauthorization is scheduled for the 102d Congress) 
and the Science and Engineering Equal Opportuni- 
ties Act to add provisions that address diversity in 
research and science education funding, and empha- 
size undergraduate teaching opportunities at certain 
categories of institutions such as historically Black 
colbges and universities (HBCUs).^ Programs 
targeted to U.S. minorities, women, and the physi- 
cally disabled could help to expand the pool of 
potential scientists and engineers. It is clear that in 
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■■Ofnce of l\eclu)ology AMCMmcnt, op. cit., footnote 74. p. 101. 

*^or the Kope of current provUioiu, tec Margot A. Schenet, Congreuioiul Research Service, ' 'Higher Education: Reauthorizatioo of tiie HlgtMr 
Educatioo Act," Iifue Brief. May 13, 1990. and PubUc Law 96-316, 94 SUL 3010. Section n, Science aoJ Bngineering Equal Oppommitie* Act, Part 
S. as amended l>y PubUc Uw 99-159, 1982. 
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this particular realm of human resources, market 
forces alone will not increase the participation of 
these groups. Policy intervention is required and 
Con^ss is empowered to intervene. 

Hie capacity of the research system could also be 
augmented by encouraging "have-not" institutions 
to concentrate excellence in select research pro- 
grams (departments and centers) and build from 
there. Attempting to enter the top ranks of federally 
funded research-intensive universities through 
across-the-board enhancement of aL' research pro- 
grams may lead to each program being unable to 
garner enough support to improve research capabil- 
ity. Various programs that address geographical 
diversity, such as the NSF Experimental Program to 
Stimulate Competitive Research (EPSCoR), or 
greater consideration of geogr^hy in funding allo- 
cation within the portfolios of mainstream scientific 
merit-based programs, could build research capacity 
that benefits States and regions as well as the Nation 
as a whole. 

Research and Education in Flux 

Calls for the reform of higher education in the 2 1 st 
century are now emanating from many presidents of 
research universities.'*' These calls center on im- 
proved undergraduate education and a better balance 
between research and teaching. Many see a need to 
change the reward system of the university, since 
asking universities to augment the teaching of 
undergraduates may be misplaced if faculty continue 
to view this as a drain on time that would be better 
spent doing research.'^ 

The tension between research and teaching is 
perpetuated by the provision of funds meant to 
improve both the institution's research performance 
and teaching capability. A common perception 
during the 1960s was tlmt Federal dollars that 
supported research also benefited undergraduate 



teaching because these top researchers would com- 
municate their excitement about developments "at 
the laboratory bench" to undergraduate and gradu- 
ate students alike. In the 1980s, with the separati«'n 
between research and undergraduate education be- 
coming more pronounced, the connection between 
research progress and the cultivation of human 
resources grew more tenuous.'^ These calls for 
increased undergraduate teaching by faculty seek to 
alter an academic research and teaching model in the 
United States that is already under strain. 

TTie predominant mode of academic research in 
vxK natural sciences and engineering begins with a 
research group that includes a PI (most often a 
faculty member), a number of graduate students, one 
or several postdoctoral scientists, technicians, and 
perh^s an additional nonfaculty Ph.D. researcher. 
While this group may be working on a single 
problem funded by one or two grants, subsets of the 
group may work on different but related problems 
funded simultaneously by multiple project grants. 
(In the social sciences, the groups tend to be smaller, 
often numbering only the faculty member and one to 
two graduate students.) 

In addition, the dominant model to launch a career 
as a young scientist is movement from one research 
university to another with an assistant professorship, 
the attainment of a first Federal research grant, and 
the re-creation of the mentor's professional lifestyle 
(e.g., independent laboratory, graduate students, 
postdoctorates). For an institution to subscribe to 
this model tends to shift much of the actual 
responsibility for awarding tenure from the depart- 
ment faculty to the Federal Government. While 
university officials say there is "... no fixed time in 
which researchers are expected to become self- 
sufficient through outside grants . . . researchers 
who have failed to win such grants are less likely to 



""."f T ^ u p«rt of ihil M«„mert. Sunfoid and Michlgw. See Karen Orawmuck, "Somo 

•mil. »<»l<j lucMe gothioi leu Oiu . ndttollioo of hcully KbolinUp ttul Itaclodei ludUlu. S« Emul L Bovs Sch«l^,u„ 
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Seventh graders observe research In a "deanroom." Seeing science at work Is Important for all age groups. 



earn tenure than their colleagues who have found 
such support, 

There is doubtless a role for universities to play in 
the diversification of research careers of recent 
Ph.D.s. New Ph.D.s find it difficult to entertain 
alternative opportunities if they have no experience 
with them, llius, programs that offer a summer in a 
corporate laboratory or part of an academic year at 
a 4-year liberal arts coUcge can help advanced 
graduate students visualize working in settings other 
than the university. Anangements that link an 
HBCU or liberal arts college to a research university 



or national laboratory stretch the resources and 
experience of both participating institutions.^ 

New Models: University and Federal 

Other models of education could be encouraged 
that feature a greater sharing of resources (e.g., 
equipment and space) and people (e.g., doctoral 
students, nonf acuity researchers, and technicians). 
Models that stress research in units other than 
academic departments, research in nonacademic 
sectors, and nonresearch roles in academia could be 
entertained. Some Federal research agencies akeady 



*3Scc Dtbm E. Blum, * •Younger ^idcntiiu Feci Big Preisurc in Battle for Ortntt. ' * The ChrotUcle of Higher Education, vol 37, No. 4, Sept. 26, 1990, 
p. A16. Ai OQC rciearcber piiu it: **Le4ding univmitiea ihould make their own <lcclsk)ns about wtio their faculty are going to be, and uot leave it to 
tbe itudy aectloiu of NIH;* Quoted in David Wbecler, ''Biomedical Reaearctkcn Seek New Souicea of Aid for Young Scientisti,'* The Chronicle cf 
Higher Education, vol. 36. No. 42. July 5. 1990, p. A23. 

^Ib date, luch arrangoDenis have been moct common in undergraduate engineering. One coalition, tpeartieaded by a S-year $15 mUUoa National 
Science Foundation gimm. will eitablis^ a communication network for information diwqninatioQ, faculty exchange, woikahopi, and outreach to 
elcflientary , lecondary, and community college itudenu. The participating universitiei are City College of New York, Howard. Mar>'laad, Muuchoietts 
Institute of l^choology. Morgan State, Pennsylvania Sute, and Washiugton. Soc * *NSP Announces Muia-Million Dollar Grants to Form Engineering 
Education Coalitions.* ' NSf News, Oct. 9, 1990. 
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recognize the development of this fonn of teamwork 
in their funding programs and support of the research 
infrastructure. For exan^le, these models are institu- 
tionalized in the centers programs sponsored by 
NSF. Centers, which support individual researchers 
(as faculty and mentors) as weU, may represent a 
new way of doing business for NSF. Centers are also 
featured at NIH intra- and extramurally; at the 
laboratories affiliated with DOD, E>OE, and NASA; 
and at the agriculniral experiments stations funded 
through block grants by USDA.'^ 

Research in general is becoming increasingly 
interdisciplinary, i.e., it requires the meshing of 
different specializations to advance a research 
area.^ Academic departments house specialists by 
discipline whose research will be performed in 
units — centers, institutes, programs — that cut across 
the traditional departmental organization on cam- 
pus. Such organized research units have a history on 
U.S. university campuses, but not as dominant 
structures.^ However, as outlined above, in many 
fields there is movement toward an industrial model 
of research, characterized by larger research teams 
and a PI who spends more time gathering fiinds to 
support junior researchers who in turn devote their 
full time to research. For many of today's research 
activities, this model seems to enhance productivity 
and allow more complex research problems to be 
tackled, by specializing responsibilities within the 
research team and sharing infrastructure.^^ 

The expanding size and complexity of research 
teams under the responsibility of entrepreneurial Pis 
and "lab chi?fs" fosters financial and organiza- 
tional strains. To help ease the strains caused by a 
transition in some parts of the research community 
to an industrial model, the Federal research agencies 
could encourage alternative models of education-in- 



research that feature a greater sharing of resources 
and people. While it is not the role of the Federal 
Government to dictate university research or educa- 
tion policies, it can provide the impetus "ot examin- 
ing and experimenting with those policies through 
grant support. 

Mainstream agency programs have always 
awarded research funds to advance the state of 
knowledge in their programmatic areas mainly on 
the core criterion of "scientific merit." Though 
difficult to define precisely, this is generally taken as 
a necessary condition for fiinding. Recognition that 
discipline-based agency programs favor investigator 
track record in proposal review, but that other factors 
reflect imoortant objectives of research funding, led 
to the creation of set-aside programs. These pro- 
grams, originating both in Congress and within 
agencies, restrict the competition for scarce funds 
according to some characteristic of the investigator 
or the proposal. Set-aside programs thus evaluate 
proposals first and foremost on scientific merit, but 
redefine the playing field by reducing the number of 
competitors. (Examples discussed in the full report 
include NIH's Minority Biomedical Research Sup- 
port Program; NSF's aforementioned EPSCoR, 
Presidential Young Investigator, and Small Grants 
for Experimental Research programs; and the Small 
Business Innovation Research programs conducted 
by various Federal agencies.) 

Taken together, such programs address the com- 
petitive disadvantage faced by young, minority, or 
small business research performers; by researchers 
and institutions in certain regions of the Nation; and 
by ideas deemed ' 'high-risk ' ' by expert peers or that 
do not fit with traditional disciplinary emphases. 
The proliferation of such programs over the last 20 
years has been a response to the desire to enlarge 



tA« ^^^^i^,^,"^ Foundation lupponed 19 Englncertng Research Ctaten and 1 1 Science and Technology Reteuch Cenlen (STCi) 
at $*8 million and $27 ,aU lion, reipectlvely. Tltu,. together they account for leu than 10 percent of the National SclenceFouSuSn-rbuSi ww2 
^ovldlng a long-tcnn f,„Kling b«e (5 to 1 1 ye«.) for IniadlscipUnaiy and high-risk projccUoriented ,o the appUed. (tevetopTHS? 
end of the rewarch contuwum. See Joacph Palca and Biiot Manhall. "DIoch Leaves NSF in Mainitreaaj,' ' Science, vol 249 Au« 24l«oT830 
^ ^^SL'^: r'"-'^'"^*"^ ^ "C; • -NSF ha. rol'M the dice oo J^m^lir^cZc^LTwitiiJ^ Z> 

"^"^ ' " ^ '"^^ "^^^ ^«» ^ ^ Science, vol. 23Uan. 4. IwT.^sS 

E. Ka*h. OoMing Ihe Douidanei of DiicipUne*." Engineering EducaHon. vol. 78. No. 10. November 1988. pp. 93-98. 

n,a7S^J^^^^^"^i^ f t '^'"'^f'' "f"" Sitady-Siaie University (Boulder. CO: Weitview Pre«j. 1982). Three modeli of 

orgaau«dre*eaTrhunltt(whlch arc common in industry and thtPcdei^ 

resource, resea^h centas. and engineering lesearch ceoten. Sec Robert S. Prtedman and Rcoee C, PriednTSe WtaS^L Tte^^ 
lnAcademe."/'ofto5furf(Vj/o«r/uj/.vol. 17.ftll 1988.pp.43-61. awcuw /uncncan siyie. inree Casea 

.r ^'*'"?*' ^' ^ ^ "'^ "cidcmic model of faculty-mcntof plus graduate sfudeut U today 

hi^";J^S,!^n='^^^^ '"^'^ shaHng-wha. has long been common.^or example. ^uoJ^L^IZ 
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both the participation in, and the capacity of, the 
Federal research system. But because the annual 
funding for each program remains modest (typically 
in the $10 million range), program impact is limited. 

Without set-asides, the Federal Government 
would have little confidence that once scientific 
merit has been demonstrated, other differentiating 
criteria would be applied to the funding of research- 
ers. However, to a research system already strapped 
for resources, tlie funding of .such ''tangential" 
concerns is seen by some as diverting precious 
dollars away from the core need to advance knowl- 
edge.^^ 

Human resources aie perliaps the most important 
component of the research system. Through support 
of scientists and engineers, graduate students, and 
the educational pipeline, the Federal Government is 
instrumental in the creation of a strong research 
work force, which has been expanding under this 
support since the 1950s. In the 1990s, however, the 
research work force — in its myriad forms of organi- 
zation and scale of effort — has reached such a size 
that it feels strain under the Federal Government's 
present approach to supporting the conduct of 
research. In addition, accommodating to an expand- 
ing research work force, and to the changing ethnic 
and racial composition of students in the educational 
pipeline for science and engineering, poses chal- 
lenges to the Federal research system. Human 
resources issues have implications not only for the 
number of participants in tlie research work force, 
but also for the character of the research that new 
entrants automatically bring to the Nation's research 
enterprise. 

ISSUE 4: Retining Data Collection and 
Analysis To Improve Research 
Decisionmaking 

Summary 

Data collected on the health of the Federal 
research systenn — dollars spent for research, 
enrollments, and academic degrees awarded in 
specific fields, and outcome measures such as 
publications and citations — are extensive. In 



other areaSt however, data are scarce. For 
instance, almost no consistent infoimatioii 
exists on the size and composition of the 
research work force (as opposed to the total 
science and engineering work force), or what 
proportion is supported by Federal funds 
(across agencies). 

Most research agencies, with the exception 
of NSF and NIH, devote few resources to 
internal data collection. Consequentlyi most 
analyses must rely on NSF and NIH data and 
indicators alone, potentially generalizing re- 
sults and trends that might not apply to other 
agencies. Furthermore, it is not clear how 
agency data are used to inform research deci- 
sionmaking, as some challenge current policy 
assumptions and others are reported at inappro- 
priate levels of aggregation. 

OTA suggests additional information that 
could be collected for different levels of 
decisionmaking, concentrating in areas of pol- 
icy relevance for Congress and the executive 
branch. However, better information may not 
be cost-free. The idea is not merely to add to 
data collection and analysis, but to substitute 
for current activities not used for internal 
agency decisionmaking or external account- 
ability. Refmed inhouse and extramural data 
collection, analysis, and interpretation would 
be instructive for decisionmaking and manag- 
ing research performance in the 1990s. 

Discussion 

Many organizations collect and analyze data on 
the research system. First and foremost is NSF, with 
its numerous surveys, reports, and electronic data 
systems that are publicly available. Certainly the 
most visible conipendiimi of data on the research 
system is the biennial report, Science & Engineering 
Indicators (SEI), issued since 1973 by the National 
Science Board, the governing body of NSF,^^ Other 
sources include the other Federal research agencies; 
the National Research Council; the Congressional 
Research Service; professional societies, especially 
the American Association for the Advancement of 




'^Change comes iiicreniemally and at the niargias of the cnlcrpris<.v Bui if oiic were constructing the system from scratch, mains tr earning crilcm to 
reflet I ihc multiple objectives of research funding would b« a key element to consider. 

'^Scc Susan li Coz/ens. **Saence Indicators. Description or Prescription?'* OTA contractor rt?pori. Scptcmlvrr 1«>9(). Note that Snt^nce <i 
Kngineetinfi Indicators (SHI) was named Saenct Indicators uJinl 1987. Sill builds on data collected, published, and issued in many other rcpurts 
by the Science Resources Smdiei Division of the National Science Poundution. 
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Science; and other public and special interest 
groups.*°* 

Ibgether these databases and analyses provide a 
wealth of information: time series on the funding of 
research and development (R&D); expenditures by 
R&D performer (e.g., universities and colleges, 
industry, Federal laboratories), by source of funding, 
and by type (basic, applied, or development); 
numbers of students who enroll in and graduate with 
degrees in s/e; characteristics of precollege science 
and mathematics programs and students in the 
education pipeline; and si2f\ sectors of employment, 
and activities of the s/e (especially Ph.D.) work 
force.>02 Detailed analyses of the Federal budget by 
research agency are available each year, and impacts 
on specific disciplines and industries can often be 
found. 

These publications provide a basis for under- 
standing the Federal research system. But even with 
each of these organizations devoting significant 
resources to the collection of information, better data 
are needed to guide possible improvements in the 
system. with its establishment, NSF was legisla- 
tively authorized as the Federal agency data liaison 
and monitor for science and technology.'^ ^an 
be used to monitor, evaluate, anticipate, and gener- 
ally inform decisionmakers— both within agencies 
and within Congress. Although many data are 
aheady collected, they are :arely matched to policy 
questions. Other (or more) duta could improve 
decisionmaking. 

Information for Research Decisionmaking 

OTA defmes four categories of data that could be 
useful in decisionmaking: 1) research monies— how 
they are allocated and spent; 2) personnel- charac- 



teristics of the research work force; 3) the research 
process^ow researchers spend their time and their 
needs (e.g., equipment and communication) for 
research performance; and 4) outcome*— the results 
of research. Besides the considerable gaps and 
uncertainties in measures of these components, the 
most detailed analyses are done aUnost exclusively 
at NSF and NIH, and not at the other miyor research 
agencies. These analyses may not generalize 
across the Federal research system. Comparable data 
from all of the agnncies would be very useful to gain 
a more well-rounded view of federally supported 
research. 

Perhaps the most fundamental pieces of informa- 
tion on the research system are the size, composi- 
tion, and distribution of the research work force, and 
how much is federally fiinded. Varying defmitions 
pose problems for data collection and interpretation 
(for an example, see box C). These data are 
important to understand the health and capacity of 
the research system and its Federal components. In 
addition, there is evidence that research teams are 
changing in size and composition. This trend is also 
important to measure since it affects the fonn and 
distribution of Federal funding. 

Second, information is needed on expenditures 
(e.g., salaries, equipment, and indirect costs) in 
research budgets; for all research performers— 
academia, Federal laboratories, and industry; and by 
subfield of science and engineering. Data on how 
Federal agencies allocate monies within project 
budgets could also be compiled, and would illumi- 
nate how funding decisions are made within the 
research agency and would help to clarify funding 
levels in specific categories of expendihires. Better 
cost accounting and forecasting for megaprojects is 
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Box C--HOW Many ''Scfcntisto'' An There? 
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surely needed. Continuous upward revisions of cost 
estimates for megaprojecis disrupt decisions about 
tlieir future funding priority. 

Third, data on tlie research process could be 
improved in amount and kind. One trend (mentioned 
above) that OTA has noted, mostly with anecdotal 
evidence and inferences from analyses of expendi- 
tures, is the increasing size of research groups, both 
within the university structure and through Federal 
support of centers. This trend has policy implica- 
tions for the cost of research, its interdisciplinary 
capabilities, the changing demographics of the work 
force, and the aspirations of young researchers. It 
also reflects how researchers may spend their time. 
More data on * 'production units'' in research, and 
their dependence on Federal funding relative to other 
sources, would augment enrollment, Ph.D. award, 
and work activity data. Gianges in the structure of 
production units have also influenced the research 
process and tlie volume — and perhaps the charac- 
ter — of outcomes.*^ Inl'ormation on the research 



process would yield a firmer foundation on which to 
base funding allocation decisions, specifically: 1) 
how researchers spend their time, 2) movement of 
research teams toward a more industi ial model in the 
allocation of responsibilities, 3) changing equip- 
ment needs and communications technologies, and 
4) requirements and average time to attain promo- 
tions in the scientific work force. 

Evaluating Research Outcomes 

Because of the fundamental and elusive nature of 
research, measuring its outcomes — in knowledge 
and education — is very difficult. The most elusive 
outcome is cultural enrichment — the discovery and 
growth of scientific knowledge. As 0MB Director 
Richard Darman has said (speaking of the proposed 
Moon/Mars mission): '*No one can put a price on 
upliftmg the Nation.'' Research has resulted in many 
benefits and is funded precisely for this reason. Tliis 
kind of benefit is nearly impossible to measuie. 
However, there are some proxies. 



•^lljc role of bboraiory cliicf or icaiu leader combines eotrepreneiinal and adinjmstralivc/supervi.sory tasks Both are essential to liic funding and 
lougcviiy of ilic puxiuciivc research unit. On the emergciKc of the enircprcncunal role on campu.s. see El/lowia. op. cii.. foomote 61. 
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When looking at research as a contribution to 
education, numbers of degrees can be tallied and 
assertions made about sldUs added to the Nation's 
work force. When looking at research as creating 
new knowledge, one tangible "output" is papers 
published by scientific investigators to communi- 
cate new information to their scientific peers. 
Communicating the results of scientific research to 
colleagues through publication in the open literature 
is considered to be an important feature of good 
research practice.*^ Peiiiaps the best approach is to 
construct workable indicators and include a rigorous 
treatment of their uncertainties. 

One tool that h'\& been vigorously developed for 
measuring the outcomes of research is bibliomet- 
rics — the statistical analysis of scientific publica- 
tions and their attributes. ^''^ Intrinsic to scientific 
publication is the referencing of earlier published 
work on which the current work is presumably based 
or has utilized in some way. References are a 
common feature of the scientiHc literature, and by 
counting how often publications are cited, biblio- 
metrics can arrive at a weighted measure of publica- 
tion impact — not only whether publications have 
been produced, but also what intact those publica- 
tions have had on the wo^k of other scientists.'°^ 

OTA has explored s^,veral examples of new data 
sets that could be co.npiled using bibliometrics."^ 
First, universities ran be ranked according to an 
output or citation measure— the citation rates for 
papers authored jy faculty and others associated 



with each institution.*^* Institutions can be ranked 
by total number of cited pi^)ers, the total citations 
received by all papers associated with each institu- 
tion, and the ratio of number of citations to the 
number of publications, namely, the average cita- 
tions per cited paper. This appears to be a more 
discerning measure than eitho: publication or cita- 
tion counts alone. 

For example, a ranking of institutions by average 
citation rates can be used in conjunction with the list 
of top liversities, in Federal R&D funding re- 
ceived, io link inputs with outputs. Together, these 
measures illuminate oifferences in rank.**^ Not only 
can publishing entities be analyzed, but so can fields 
of study. For instance, "hot fields,'* in which the 
rate of publication and citation increases quickly 
over a short iod of time, can be identified and 
"related fields," in which published papers often 
cite each other, can be mapped.**^ Because of 
problems of interpretation in bibliometric analysis, 
it should be seen as "value-added" to reserach 
decisionmaking, not as stand-alone information. 
Bibliometrics could be used to help monitor out- 
comes of research, e.g., publication output and oth^ 
information from the research system.''^ 

Criteria that go beyond bibliometric data could be 
specified for such evaluations. These criteria could 
include the originality of research results, the 
project's efficiency and cost, impacts on education 
and the research infrastructure, and overall scientific 
merit. Such research project evaluation could be 
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productionof inferior research. For discussion, seech. 5 of the full OTA report. u.*,^) m wk> 

iiipot example, sec AngeU Martello. "OovenjmcnU Led in Funding 1989-90 'Hot Papcn* Research." The Scientist, vol. 4, No. 16. Aug. 20. 1990. 
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A Research Trlanglo Park scientist aooesses a computer 
network. Computers can grealty enhance data collection 
and presentatior. 

employed to augment agency decisions on funding 
and administration of research programs, (Some 
research agencies already utilize certain aspects of 
research program evaluation.^ 

Utilizing Data for Research Decisionmaking 

In a policy context, information must be presented 
to those who are in positions to effect change by 
allocating or redirecting resources. In the diverse 
structure of the Federal research system, research 
decisions are made at many levels. For example, an 
agency program manager requires data specific to 
the purview of his/her program, while 0MB and 
OSTP must be aware of trends in science that span 
broad fields, institutions, and agencies, as well as 
those that apply only to specific fields, performers, 
and research sponsors. 



Drawing on NSF expertise as the possible coordi- 
nating agency, information could be collected at 
each agency on proposal submissions and awards, 
research expenditures by line items in the budget, 
and the size and distribution of the research work 
force that is supported (including the funding that 
this work force receives from other sources). Infor- 
mation must be available to decisionmakers for 
evaluation as well as to illmninate significant trends. 
Often data can be presented in the form of indica- 
tors, e.g., comparisons between variables, to suggest 
patterns not otherwise discernible. NSF has pio- 
neered and sustained the creation of indicators for 
science policy and has recently suggested monitor- 
ing several new indicators (e.g., indicators of pro- 
posal success rates, PI success rates, and continuity 
of NSF support).^ 

OTA agrees that new indicators could be very 
useful, and alwSo suggests elaborating them. These 
could include measures of the active research 
community (which would calibrate the number of 
researchers actively engaged in research), and pro- 
duction units (which would track trends in the 
composition of research teams by broad field and 
subfleld). 

The combination of such indicators would give a 
more precise estimate of the changing parameters of 
the Federal research system.^ This information 
would be invaluable to policymakers concerned 
about the health of certain sectors of the system, lb 
produce such infomiation, as part of ongoing agency 
data collection and NSF responsibilities for collation 
and presentation, extra resources would be needed 
(at least in the near term). Over time, plans could be 
developed to streamline NSF data and analysis 
activities, such as a reduction in the number of 
nonmandated reports issued annually, or expansion 
of its inhouse and extramural ''research on re- 
search.*' The idea is not merely to add to data 
collection and analysis, but to substitute for cunent 
activities that are not used for internal agoncy 



1 » *For cxMnplc, icc U,S , Dcpartroenl of Energy, Office of Progmn Analysis, Office of Enagy Research, A/i Assessment of the Basic Energy Sciences 
Program, DOE/ER-0123 (Washington, DC: 1982). For t review of other evaluitioos, see National Academy of Science, Committee on Scioice. 
E«giiic«ring, and Public Policy. The Quality of Research in Science (Washington. DC: National Acadany Preas. 1982), app, C, 

i>«NaUonal Science Foundation. '*NSF Vital Signs: Troids in Research Support, Fiscal Yean 1980-89.*' draft report. Nov. 13, 1990. Some of the 
iodlcaton repotted here were used for an inhouse National Science Foundation evaluation of waya to streamline the workload of program staff and the 
external research community. See National Science Foundation, Report of the Merit Review Task Force, NSF 90-1 13 (Washington, DC: Aug. 23. 1990). 

n7por example, what would be the indications that growth in researcti productivity is slowing or that the size of t r^earch communit>* is ptfcarloosly 
large or small relative to the resourcea supporting it? See Colleen Cordes. * 'Policy Experts Aak a Heretical (^esdon: Has Academic Science Grown 
Tbo Big?** The Chronicle Higher Education, vol. 37, No, 3. Nov. 21. 1990, pp. Al. A22, 
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decisionmaking or external accountability. If 
there is a premium on timely information for 
research decisionmaking, it must be declared (and 
funded as) a Federal priority. 

Congress could instruct every research agency to 
develop a baseline of information, direct NSF to 
expand its focus and coordinating function for data 
collection and analysis, and direct OSTP (in con- 
junction with 0MB) to devise a plan to increase the 
reporting and use of agency data in the budget 
process, especially crosscutting information in pri- 
ority research areas. Using the FCCSET mechanism, 
this has already been done for global change, 
high-performance computing, and most recently, 
science and mathematics education.^ This mecha- 
nism seems to work and could be more widely 
emulated. 

In summary, better data on the Federal research 
system could be instrumental in the creation or 
refinement of research policies for the 1990s. (For a 
summary of data oriented to different users, see table 
5.) The utility of data, of course, is judged by many 
participants in the system: the needs of Congress are 
usually agency- and budget-specific the agen- 
cies, in contrast, v/orry about the performance of 
various programs and their constitutent research 
projects. While data collection by NSF and groups 
outside the Federal Government has been instruc- 
tive, it could be greatly enhanced. Much information 
could be collected on the Federal research system 
that maps trends, at different levels of aggregation 
and units of analysis, for different users. However, 
the existence of data does not ensure their utility. 



The highest priority in data collection for research 
policymaking in the 1990s is comparable data from 
ail of the agencies to help Congress maintain a 
well-rounded view of federally supported research. 
The second priority is data presented in forms that 
are instructive at various levels of decisionmaking. 
New data and indicators^ grounded in the tradition of 
die SEI volumes and extramural research on re- 
search, are needed to monitor changes in the Federal 
research system.^^^ Finally, OTA finds that research 
evaluation techniques, such as bibliometrics and 
portfolio analysis, cannot replace judgments by 
peers and decisionmakers, but can enrich them. 
Ongoing project evaluation could keep agencies 
alert to changes in research performance and aug- 
ment program manager judgments about performers 
and projects. In short, such evaluation could serve to 
improve overall program effectiveness. 

One of the functions of analysis is to raise 
questions about the information that decisionmakers 
have at their disposal, to assess its advantages and 
disadvantages, and to define a richer menu of 
options.^^^ Improving the measurement process 
could help to quantify existing opportunities and 
problems, and pinpoint previously uncovered ones, 
relevant to decisionmaking at all levels of the 
Federal Government. 

Toward Policy Implementation 

Since the post-Sputnik era, both the U.S. capacity 
to perform research and the demand for funds to 
sustain scientific progress have grown. Federal 
investments have fostered the research system, 
managed through a pluralistic agency structure. This 
struclwe has supported the largest and most produc- 



1 1 ^The NadonaJ Science Foundttion routinely conducts * *wscr lurveyi. ' ' If Science Resources Studies (SRS) knows from questioDnalre responses bow 
ib vadouA data reports arc used— do they influence research or education policies? are they a source for administnton or faculty-researcben?— then 
NSF should have a leoie of sudience "consumption" and "utilization** panerns. These would suggest which reporu couJd be dropped, replaced, and 
modified. For an example of the SRS inventory of "intramural publicationi." see National Science Foundation, Publications List: 1977-1987, NSF 
87-312 (WaihingtoD, DC: July 1987). 

1 1'OTA interview with Office of Management and Budget staff. Feb. 7, 1991 . 

i^As several National Science Foundation staff have indicated to OlA project staff (personal communications, October-December 1990), the Science 
Advisor draws heavily on unpublished and newly publiihcd Science A Engineering Indicators (SEI) data in preparing and presenting the 
Administration's policy proposals at congressional "posmre hearings" early in the annual authorization proceu. Indeed, the production cycle of Sfil 
is geared to delivery of the volume as an input to this budget process. 

^3 'Quantitative dau will not fu0ice. Information on the contexu in which research is performed, and characteristics of the performers individually and 
collectively, will provide clues to bow the numbers can be Interpreted and perhaps acted on. For example, see Daniel T. Layzell "Most Research on 
Higher Education la Stale, Irrelevant, and of Little Use to Policymakers, ' * The Chronicle of Higher Education, vol. 37. No. 8, Oct. 24, 1990. pp. B 1, 
B3. 

i^^This leads OTA to suggest that the research agencies, especially the National Science Foundation and its policy programs, renuiin in close touch 
with external analyu of the Federal research system. Keeping abreast of other new measurement techniques and findings related to people, funding, 
and research activities would be a modest but fruitful investment in extending inhouse capabilities and refming knowledge of federally sponsored research 
performance. 
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Table S-Dealrad Data and Indlcatora on the Federal Research System 



Primary users 



Category 



Description 



Method 



Congress Agencies 0MB OSTP 



Agency funding 
allocation method 



Recoarch expenditures 



Research workforce 
Research process 



Outcome measures 



Indicators 



Funding within and across fields and 
agencies 

Cross-agency Information on proposal 
submissions and awards, research 
costs, and the size and distribution of 
the research work force supported 
Research expenditures In ecademia, 
Federal and industrial laboratories, 
centers, and university/Industry col- 
laborations 
Agency allocations of costs within re- 
search project budgeU, by field 
Megaproject e)q)endltures: their compo- 
nents, evolution over time, and con- 
stmctlon and operating costs 
Size and how much is federally hinded 
Size and composition of research groups 
Time commitments of researchers 
Patterns of communication among re- 
searchers 

Equipment needs across fields (Including 

the fate of old equipment) 
Requirements for new hires in research 

positions 

Citation impacts for institutions and sets 

of institutions 
international collaborations In research 

areas 

Research-technology interface, e.g., uni- 
versity/Industry collaboration 

New production functions and quantita- 
tive project selection measures 

Comparison between earmariced and 
peer-reviewed project outcomes 

Evaluation of research pro|eot8/progranr^ 

Proposal success rate, PI success rate, 
proposal pressure rates, flexibility and 
continuity of rapport rates, pn>lect 
award and duration rate, active re- 
search community and production 
unit Indices 



Agency data col- 
lection (c^nd 
FCCSET) 



Agency data 
cdlectlon 



Lead agency 
s jrvey 

Lsafl agency 

I jrve)*; onsite 
studies 



Blbllometr1c&; 
surveys of 
lr>dustry and 
academia 



X 

X 



Agency analysis 



Ri^??5cSEUF«der«l Coofdlneting CouKil on Sd«nce. Enolnetring. .rni Technology: OMB-Oftice of Maneginient and Budget; OSTP-atloi of Sdence 

end Technology Policy; Pi-pflndpei Inveitigator. 
SOURCE: Office of Ttchno<ogy Awewment, 1W1. 



tivc research capability in the world. For many 
decades, scientific research has contributed in im- 
portant ways to the cultural, technological, and 
economic base of the Nation. 

In the 1990s, changing funding patterns and 
various pressures from both outside and within the 
scientific community will test the Federal research 



system. In such an environment, the prospects of 
fashioning a system that is responsive to national 
needs through selective, yet generous research 
funding will demand well-informed, coherent poli- 

123 



cies/ 



The system will face many challenges, but four 
are clear: First, new methods of setting iwiorities and 



•J^Aj Brooks hM ob»ved: * 'TTk rcKerch cmeqjriie U inorc like M 
the functioiiing of the whole orgtnim and not Just the pertlcokr ftmction oiteotibly lovcd by the ptit rtmoved." Hirvey Brooki. * •ModcU for Science 
PUnniafi,'* PMic Administration Review, vol. 31. May/June im, p. 364. PoUclea must reapood to. tad in tome wtyi, tiUkipftte. the cooaeqocooes 
of fundintdccijiooi on the rcie«rcbsynem. Indeed, thiff^ to manage the futore of the lyiteni. 
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increased use of existing methods are required at ail 
levels of decisionmaking. Second, Federal expendi- 
tures for individual con^nents of research projects 
have increase^l faster than inflation. Understanding 
and coping with these increases is imperative in 
research decisionmaking. Third, the development of 
human resources for the science and engineering 
work force must occ jr through Federal incentives 
and institutional programs that act on the educa- 
tional pipeline (K-12 ttirough graduate study). Fi- 
nally, gaps and uncertainties in the data used to 
describe the Federal research system must be 
reduced, and be replaced by more routine provision 
of policy-relevant information. 

OTA finds that Congress^ the executive 
branch, and research performers must converge 
on these issues. Potential congressional actions 
fall into three categories. Congress can: 1) retain 
primary responsibility for decisions and initiat- 
ing actions; 2) place some of the responsibility fr r 
coordination and decisions on the executive 
branch; and 3) encourage research performers 
(especially universities, as well as Federal and 
inG'istrial laboratories) to address components of 
the^e issues. (For a summary of possible actions, see 
t^le 60 

At the congressional level, heaniigs, legislation, 
and oversight should first address crosscutting and 
within-agency priority setting at the national level 
OTA suggests that one or more committees of 
Congress routinely (preferably biennially) hold 
hearings that require the research agencies, OSTP, 
and 0MB to present coordinated budget plans with 
analyses that cut across scientific disciplines and 
research areas. Coordination among relevant com- 
mittees of Congress would make this most produc- 
tive. These hearings could also focus on crosscutting 
criteria for research decisionir^aking within and 
across agencies. Emphasis mus \ Jt pi acrxi on criteria 
to expand the future capabilities of the research 
system, such as strengthening education and hur.ian 
resources. A second set of congressional actio.'^s 
could explore cost-accountability effort.^ at the 
research agencies and throughout the research sys- 
tem. A final set of hearings ouglu to cxaxnine the 
state of data on the research system and improve- 
ments to infomi congressional decisionmaking.'^ 



Tabit 6--^mmary of PoMlbit CongrtMloncIt 
Executive Branchi and RtMarch Parformar Actions 



CongrwiQfml tmrlngB, 
I9gl$latl¥0 %ttom, 
end overBlght to: 



EM0Cutlv0 bmnch 
actions to: 



Research performer 
act tons to: 



» Set prtoritles across and within 
agendes, and dvvelop appropri* 
ate agency missions. 

> Evaluate the total portfolio to see 
If ItfulflRsnatlonalrssearoh goals, 
human resouroes needs, soten- 
tlf io Inf rastnicture developnient, 
and balance. 

Initiate greater oost-aooounta- 
billty throughout the Fedora! 
search systenx 

Expand programs that fortify the 
educational pipeline for soienca 
and engineering, and monitor the 
oomt)lned oontrtMtlons of 
agency programs to achieve edu- 
cation and human resources 
goals. 

Auomentdataandanalysisonthe 
Federal research system for con- 
grssslonal decisionmaking. 

Enhance cross-agency prtortty 
setting In the Federal budget and 
Inaease research agency f lextbll- 
Ity to address new priofltles. 
Institute better cost-aooount- 
ability and cost-containment 
measures by agendes and re* 
search performers. 
Expand agency programs to pro- 
nx)te partldpatton In the educa- 
tional pipeline for science and 
engineering, and require agen- 
das to report progress toward 
these goals. 

Monitor and analyze poitcy- 
reievant trends on the research 
system, especially as related to 
the changing organization and 
productivity of research groups 
and Institutions. 

ContHn ar>d account for research 
oxpendltures. 

Revise education and research 
poildes as they affect: a) reoult- 
ment and retention In the educa- 
tional pipeline for science and 
engineering, and b) faculty pro- 
motion, tenure, and laboratory 
practices. 



SOURCE: CHik» of Technology AM«ftam«nt 1091 . 



Hearings could be followed with cor sessional 
oversight — on agency progress toward their re- 
search missions, implementing the criteria chosen 
by Confess to enhance research decisionmaking, 



li^Thert U t role for the congrcasiotuJ fupport agfocict. u well as other sources of cjtpeit advice. For other propostla. ice Ctroegie Commijidoo oa 
Technolojjy. ukI GoverDmcai, Sdeitc^, Technohsy. and Congrtu: Expert Advict and the Decuhnmaking Proc€4S (New York, NY: Febnuxy 



ERIC 



1:; 



42 • Federally Funded Research: Decisions for a Decade 



instituting greater cost- accountability, and provid- 
ing useful data and analysis on an ongoing basis to 
Congress. 

Some of these hearings and oversight efforts 
akead y take place in committees of Congress. While 
they have been very useful, OTA fmds that to effect 
change in the rercarch system, congressional action 
must be comprehensive and sustained. Posiure 
hearings with the Science Advisor and agency 
directors will not suffice. 

In iis role as the prime sponsor of Federal research, 
the executive branch (especially OSTP, 0MB, and 
the research agencies) could provide more flexibility 
in response to changing researci. priorities. For 
instance, the executive branch could systematically 
initiate tradeoffs among agency research programs 
including, with the cooperation of Cong ^-^s, the 
terrunation of programs. This would help tu create 
more coordinated research policies. Similarly, the 
re.search agencies could institute greater cost- 
accountability measures, and include costs as ex- 
plicit factors in decisionmaking at the project level. 
This would provide a more realistic assessment of 
future capabilities with respect to projected funding 
levels. On human resources issues, the executive 
branch could implement or expand agency programs 
and reporting requirements to: 1) encourage recruit- 
ment and retention of women, U.S. ^ninorities, and 
other underparticipatmg groups in the educational 
pipeline for science and engineering; and 2) monitor 
the changing structure of research performance, 
especially forms of research organization, and de- 
vise fimding allocation methods that accr imodate 
both the needs of the PI and research tear finally, 
each of the research agencies (with NSP as the lead 
agency) could conduct routine data co) .ection ana 
analysis on policy-relevant aspects o^t^ ^ir program- 
matic conuibutions to the reseai c^: f ^tem. 

Not all problems in the research ; .v^m, however, 
can be addressed in Congress or by the executive 
branch. Universities and laboratories (bo*h Federal 
(u;H industrial) are key components of the system, 
and many policies are dictated by the practices 
wiihiii these institutions. Containing research ex- 
penditures and expanding the educational pipeline 
through institutional programs and requirements are 
examples of policy areas in which ^search perfomi- 
ers must fulfill their role in the social contract 
implied by the Federal patronage of research. The 
Fcdcr:;.! Government can only encourage univcrsi- 




Communication among sdentlsta and enginoers Is an 
essential pr^rt of the resear^^ process. 

ties and laboratories to follow new paths; fev/ iiiect 
Federal incentives are available to initiate change. 
Greater delineation of govertunent and reseiLrch 
performer responsibili'ies would help to sanction 
congressiona' and executive branch action on prob- 
lems in the research system. 

Ir addition to specifying at which level (ccngres- 
sional, executive branch, or research performer) 
issues could be app opriatel) addressed, responses 
to the four challenges ouuined above must also 
recognize many inherent tensions in the research 
system. They include the merits of more centralized 
decisionmaking juxt^osed against the advantages 
(and realities) of a decentralized Federal research 
system. Other tensions arise b<itween the funding of 
mainstream individual investigator programs and 
se' aside or more rpecialized programs (see again 
table 1). Inevitably, policies that relieve some 
t'^nsions will engender others. 

In summary, decisionmaldng in the Federal le- 
search system concerns many laiulabie goals, f nd 
the options arc clearly competing ••goods." Thus, 
the Federal Government must make ough choices, 
even beyond issues of merit and constricted budgets, 
in g ilding the research system. A quarter-century 
ago, a chapter on Science and the Federal Govern- 
ment'' concluded with these words: 
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As never before in history, the status of science 
and technology has become an important hallmark of 
a nation's greatness; and the United States clearly 
has perceived and acted upcn this fact. In the process, 
the Federal Government nas displaced the univer- 
sity» industry, and the private foundation as chief 



patron and has fashioned a host of institutions to 
administer vastly increased commitments to scien- 
tific and technological excellence. 

Sustaining and managing this system is the chal- 
lenge of the decade ahead. 



I 
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Office of Technology Assessment 



The Office of Technology Assessment (OTA) was created in 1972 as an 
analytical arm of Congress. OTA*s basic function is to help legislative policy- 
makers anticipate and plan for the consequences of technological changes and 
to examine the many ways, expected and unexpected, in which technology 
afTects people's lives. The assejsment of technology calls for exploration of 
the ph>sical, biological, economic, social, and political impacts that can result 
from applications of scientific knowledge. OTA provides Congress with in- 
dependent and timely information about the potential effects — both benefi- 
cial and harmful — of technological applications. 

Requests for studies are made by chairmen of standing committees of the 
House of Representatives or Senate; by the Technology Assessment Board, 
the governing body of OTA; or by the Director of OTA in consultation with 
the Board. 

The Technology Assessinent Board is compo.sed of six members of the 
House, six members of the Senate, and the O TA Director, who is a non- 
voting member. 

OTA has studies under way in nine program areas: energy and materi- 
als; industry, technology, and employment; international security and com- 
merce; biological applications; food and renewable resources; health; 
telecommunication and computing technologies; (.ceans and environment; 
and science, education, and transportation. 
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